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MEMORANDUM  FOR  UNDER  SECRETARY  OF  DEFENSE  FOR  RESEARCH 
AND  ENGINEERING 

SUBJECT:  DSB  Task  Force  on  Transition  of  Weapon  Systems  from 
Development  to  Production 


Enclosed  is  the  final  report  of  the  DSB  Task  Force  on  the 
Transition  of  Weapon  Systems  from  Development  to  Production. 

This  effort  was  undertaken  to  develop  recommendations  to  improve 
and  accelerate  the  transition  of  weapons  systems  into 
production.  The  attached  report  summarizes  the  results  of  the 
various  panels  formed  to  review  this  problem  and  contains 
specific  recommendations.  This  report  has  been  reviewed  and 
approved  by  the  Defense  Science  Board. 

Additional  reports  were  prepared  by  ad  hoc  working  groups 
established  by  this  Task  Force.  They  represent  the  results  of 
considerable  effort  and  should  prove  useful ,  although  they  have 
not  been  reviewed  by  the  DSB  and  therefore  do  not  represent 
official  DSB  reports.  This  applies,  for  example,  to  the  effort 
entitled  "Solving  the  Risk  Equation  in  Transitioning  from 
Development  to  Production." 


The  Task  Force  chaired  by  Will  Willoughby  and  supported  by  the 
working  groups  put  much  effort  into  the  solution  of  this 
problem,  and  hopefully  has  produced  recommendations  that  will 
improve  the  transition  process.  As  Will  Willoughby  has  stated 
in  his  transmittal  letter  "the  key  to  success  lies  in 
implementation."  As  always  the  DSB  stands  ready  to  assist  with 
this  task. 
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IN  REPI.V  REFER  TO 

25  May  1983 


Mr.  Norman  R.  Augustine,  Chairman 
Defense  Science  Board 
Room  3-D-1034,  The  Pentagon 
Washington,  DC  20301 

Dear  Mr.  Augustine: '  1 


Enclosed  is  the  report  of  the  Defense  Science  Board  Task  Force  on 
Transition  of  Weapon  Systems  from  Development  to  Production.  The 
study  was  conducted  in  response  to  the  Terms  of  Reference  in  Under 
Secretary  of  Defense  for  Research  and  Engineering  memorandum  of  28 
June  1982  (Appendix  A  to  the  report) . 


The  effort  focused  on  a  concise  group  of  design,  test  and  manufac¬ 
turing  fundamentals,  including  key  facilities  and  managerial 
issues,  which  constitute  the  essential  elements  of  low  risk  programs. 
This  effort  produced  a  transition  document  that  covers  some  of  the 
critical  risk  areas  that  have  been  observed  in  the  process  of 
transitioning  from  development  to  production.  This  guidance  in¬ 
formation  is  totally  consistent  with  established  DoD  policy  and  is 
presented  in  the  form  of  "templates"  which  address  procedures  for 
reducing  those  risk  areas.  These  templates  are  intended  to  be  used 
by  DoD  and  industry  in  making  decisions  on  individual  acquisition 
programs . 


The  Task  Force  is  satisfied  with  the  results  of  its  efforts  and 
feels  that  the  objective  of  this  study  was  successfully  accomplished. 
However,  the  key  to  ultimate  success  lies  in  implement ation  of  the 
Transition  Document. 


Sincerely, 

W.  J.  Willoughby,  Jr. 

Deputy  Chief  of  Naval  Material 
(Reliability ,  Maintainability 
and  Quality  Assurance) 
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EXECUTIVE  SUMMARY 


3 


OBJECTIVE 


The  objective  of  this  report  is  to  provide  recommendations  that  will  lower 
the  risk  of  transitioning  weapon  systems  from  design  and  development  into 
productionv 


BACKGROUND 


) 


^The^jast  years  have  witnessed  a  rapidly  accelerating  interest  in  upgrading  the 
reliability  and  maintainability  of  the  weapon  systems,  those  already 
operational  and  those  programmed  for  ultimate  delivery  to  the  services.  Major 
reasons  for  this  increase  in  interest  include  the  length  of  the  acquisition 
cycle,  the  unsatisfactory  levels  of  effectiveness,  and  budget  pressures  to 
reduce  the  life  cycle  costs  of  new  weapon  systems.  Yet,  in  spite  of  this 
interest,  tangible  improvement  in  the  reliability  and  maintainability  of 
products  reaching  the  field  continues  to  be  an  area  of  concern.  And  a  major 
cause  for  this  concern  has  been  observed  to  be  the  failure  to  make  the  proper 
transition  from  design  and  development  into  production. 

The  Defense  Science  Board  (D5By Task  Force  on  Transition  of  Weapon  Systems 
From  Development  to  Production  was  established  in  June  1982  to  identify  the 
fundamentals  of  low  risk  design  and  manufacturing  processes.  These  fundamen¬ 
tals  were  to  Include  not  only  the  technical  aspects  of  the  design  and  manu¬ 
facturing  processes  but  also  some  of  the  key  funding  and  managerial  issues. 


APPROACH 


The  DSB  Task  Force  consisted  of  top  level  engineering,  production  and  program 
management  personnel  from  both  industry  and  the  government.  These  individuals 
all  possessed  extensive  experience  with  the  Department  of  Defense  material 
acquisition  process.  The  approach  involved  the  formation  of  five  panels  to 
address  issues  related  to  design,  test,  production,  facilities  and  investment, 
and  management,  respectively.  Each  panel  was  chaired  by  a  Task  Force  member 
and  each  member  generally  supported  more  than  one  panel.  Each  panel  addressed 
several  of  the  issues  stated  in  the  Terms  of  Reference  and  submitted  its  own 
panel  report  to  the  Task  Force  Chairman.  These  results  were  then  reviewed  and 
consolidated  and  are  submitted  in  this  report  for  consideration  by  the  Office 
of  the  Secretary  of  Defense/Defense  Science  Board. 

SUMMARY  OF  RESULTS 


The  following  paragraphs  constitute  an  overview  of  the  key  findings,  conclu¬ 
sions,  and  recommendations  of  the  study  effort.  The  material  is  divided  into 
*he  five  areas  that  were  addressed  by  the  individual  panels.  The  body  of  this 
report  provides  a  comprehensive  discussion  of  the  results,  and  the  individual 
panel  reports  are  attached  for  reference  purposes. 


Disciplines  for  the  Development  Cycle.  Analyses  were  focused  on  the  need  for 
program  requirements  to  be  completely  defined,  contractor  program  teams  to  be 
properly  staffed  with  qualified  personnel,  design  teams  to  be  furnished  with 
proper  tools,  and  schedules  to  be  realistic.  Recommendations  in  this  area 
underscored  the  need  for  DoD  services  to  ensure  that  RFPs  communicate  all  pro¬ 
gram  requirements,  that  proposals  are  reviewed  for  credibility,  that  contract 
documents  formalize  all  requirements,  and  that  care  be  exercised  in  selecting 
the  government  program  manager. 

Figures  of  Merit  for  Allocated  Design  Time  and  Cost.  Figures  of  merit  can  be 
developed  for  front-end  loading  of  funds;  for  the  allocation  of  budget,  man¬ 
power  and  elements  of  cost;  and  for  timing  and  interrelating  the  typical 
development  phase  activities.  However,  further  study  is  required  in  this 
area,  and  the  Defense  Systems  Management  College  or  a  qualified  consulting 
firm  should  be  tasked  to  continue  the  work  of  the  panel. 

Key  Predictors/ Indicators  of  Potential  Design  and  Manufacturing  Problems. 
General  indicators  of  design  problems  were  found  to  include  the  degree  and 
timing  of  change  in  design  requirements  and  the  number  of  design  corrections 
resulting  from  technical  reviews.  Indicators  of  manufacturing  problems  were 
observed  to  Include  rejection  rates  and  parts  supply  shortages.  Large  work 
force  turnovers  were  found  to  be  one  of  many  indicators  applicable  to  both 
design  and  manufacturing.  Because  of  the  importance  of  basic  indicators, 
government  programs  should  require  formal  implementation  of  a  management 
system  for  identification  and  tracking  of  critical  parameters.  The  system 
should  be  designed  to  Indicate  current  technical  status  and  to  predict  problem 
areas  requiring  corrective  action. 

Methods  of  Displaying  Design  Confidence.  Design  confidence  is  enhanced  by 
requiring  design  maturity  to  occur  in  the  development  phase,  establishing  ap¬ 
propriate  design  margins,  and  using  effective  technical  performance  measure¬ 
ment  systems.  As  a  means  for  enhancing  design  confidence,  DoD  documents 
should  promulgate  requirements  for  the  use  of  design  margins.  Implementation 
of  technical  performance  measurement  systems  should  also  be  required  by 
contract  as  a  part  of  full  scale  development  programs. 

Design  Review  Process.  It  was  recommended  that  specific  requirements  be  cited 
in  development  contracts  for  design  reviews;  that  the  review  process  be  sys¬ 
tematized,  including  the  use  of  detailed  checklists;  and  that  government  par¬ 
ticipants  should  be  selected  on  the  basis  of  experience  and  expertise.  Other 
observations  included  the  value  of  considering  major  assembly  tooling  in  the 
design  review,  the  use  of  Computer  Aided  Design  (CAD)  technology  to  support 
the  design  review  process,  and  the  use  of  an  action  item  follow  up  system  as  a 
part  of  the  design  review  process.  The  panel  also  recommended  a  policy  docu¬ 
ment  that  would  require  a  formal  design  review  plan  to  be  approved  by  the 
government  program  manager. 


Test  Panel 


Integration  of  Test.  An  integrated  test  plan  should  be  developed  early  in  the 
acquisition  process  to  ensure  maximum  efficiency  in  testing. 

Test  Environments.  One  specific  set  of  operational  environments  should  be 
established  for  the  system,  and  these  in  turn  should  be  allocated  to  each  end 
item  of  equipment.  This  does  not  preclude  the  use  of  a  more  severe  environ¬ 
ment  to  provide  a  margin  for  actual  operation  or  to  provide  failure  data. 


Reliability  Development  Tests.  Subsystem  selection  for  reliability  develop- 
ment  tests  and  the  tailoring  of  test  requirements  should  be  based  on  the  sub¬ 
system's  contribution  to  overall  system  unreliability.  Rel'  ’lity 
development  testing  should  also  be  integrated  with  other  de  opment  tests  to 
minimize  test  cost  and  the  Impact  on  program  schedules. 


Reliability  Demonstration  Tests.  Emphasis  should  be  placed  reliability 
development  and  other  development  testing  in  lieu  of  reliab  demonstration 
testing.  Dedicated  reliability  demonstration  tests  should  n  recommended 
for  development  programs.  In  special  cases  where  demonstration  is  necessary, 
the  demonstration  should  be  tailored  to  make  maximum  use  of  other  development 
and  operational  test  data. 


Application  of  TAAF  Philosophy.  Specific  DoD  guidelines  should  be  prepared 
for  defining  the  application  of  the  test,  analyze  and  fix  (TAAF)  philosophy 
during  system  and  subsystem  level  tests  and  to  provide  the  methodology  for 
reliability  growth  tracking  using  all  test  results.  Guidelines  should 
emphasize  the  need  for  a  closed  loop  failure  analysis  and  corrective  action 
program  and  for  rigorous  configuration  management  to  control  the  modification 
of  system  elements  during  the  transition  period. 

Initial  Operational  Test  and  Evaluation.  It  was  found  that  operational  test 
and  evaluation  conducted  in  discrete  phases  during  transition  from  development 
to  production  will  provide  an  operational  assessment  for  use  by  government 
decision  makers  at  appropriate  milestones.  Consistent  with  this  finding,  DoD 
Directive  5000.1  and  DoO  Instructions  5000.2  and  5000.3  should  be  modified  to 
provide  a  more  reasonable  transltion-to-production  phase  bounded  by  milestones 
instead  of  a  single  point  at  which  development  ends  and  production  begins. 

Full  Scale  Engineering  Development  Tests.  DoD  guidance  regarding  test  and 
evalutioh  should  encourage  contractual agreements  between  the  contractor  and 
the  military  services  for  military  participation  in  DT&E  during  FSED. 

Field  Operational  Performance  Feedback  During  the  Early  Manufacturing  Process. 
The  Panel  concluded  that  on-site  engineering  analysis  will  enhance  early 
correction  of  problems  encountered  in  initial  service  operation  of  new 
systems.  Therefore,  on-site  engineering  teams  should  be  provided  by  the 
contractor  to  observe  Initial  operations  and  to  assist  in  identifying  problems 
and  their  corrective  action. 


xi 
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Production  Disciplines  Panel 

Vendor  Impact  on  Quality,  Cost  and  Schedule.  Percentages  of  subcontracted 
work  on  major  weapon  systems  has  grown  and  has  reached  as  much  as  80%  in  some 
instances.  Prime  contractors  remain  responsible  for  subcontracted  work;  how¬ 
ever,  subcontract  management  plans  and  systems  need  improvement.  Also,  primes 
need  to  improve  implementation  of  their  subcontract  management  systems  to 
correct  this  area  of  concern. 

Workmanship  Defects.  Workmanship  defects  during  production  are  caused  by  man- 
agement's  failure  to  provide  proper  tools  and  resources  to  the  operators. 
Innovative  techniques,  as  described  in  the  panel  report,  are  a  means  to  cope 
with  poor  workmanship  and  excessive  rework. 

Part  Quality  Confidence  in  Manufacturing.  The  Qualified  Products  List  (QPL) 
program,  though  a  valid  concept,  does  not  currently  assure  that  parts  meet 
specified  quality  standards.  Receiving  inspections  are  generally  more  cost 
effective  than  source  inspections,  and  contractors  should  be  allowed  flexibil¬ 
ity  in  determining  and  adjusting  defect  threshold  limits  in  receiving  inspec¬ 
tion.  Contractual  leverage  for  assuring  parts  quality  include  competition, 
quantity  buys,  buyer  testing  and  feedback.  In  addition,  government  program 
managers  should  require  contractors  to  identify  their  choice  of  parts  inspec¬ 
tion  techniques  and  Include  this  information  in  the  Production  Plan.  The  gov¬ 
ernment  should  also  take  action  to  revitalize  the  QPL  into  an  effective 
program. 

Recipe  for  Entering  Production.  Government  programs  should  require  and  fund  a 
contractor  Production  Plan.  Initially,  this  plan  should  be  prepared  no  later 
than  the  start  of  engineering  development  and  be  continually  updated  until 
rate  production  Is  achieved.  A  formal  production  readiness  review  should  also 
be  jointly  conducted  before  beginning  initial  production.  Additionally, 
proof-of-design  and  proof -of -manufacturing  models  should  be  authorized  and 
funded. 

Cooperative  Participation  by  Government/ Industry.  Contractors  generally  need 
time  and  flexibility  to  optimize  unit  production  rates  and  to  solve  basic  de¬ 
sign  and  production  errors  without  government  punitive  actions.  Therefore, 
the  government  should  adopt  a  more  constructive  attitude  regarding  early  iden¬ 
tification  and  elimination  of  design  and  production  deficiencies.  Government 
control  of  the  technical  data  package  should  also  be  deferred  until  the  conclu¬ 
sion  of  the  first  year's  production  deliveries  or  IOC,  whichever  occurs  later. 

Depot  Organic  Support.  Contractors  should  maintain  control  of  the  configura¬ 
tion  for  at  least  2-3  years  beyond  deployment  to  allow  for  maturation.  During 
this  period,  the  contractor  should  be  responsible  for  the  total  analysis  and 
repair  task.  A  product  improvement  warranty  should  be  required  from  the  con¬ 
tractor  with  sufficient  incentive/penalty  provisions  for  proper  motivation. 


Corrective  Action  Systems  (CAS).  In  defense  weapon  system  contracting  proced- 
ures,  CAS  is  not  generally  recognized  and  funded.  The  consequence  is  to  delay 
the  "ready-f or-production"  date  with  attendant  schedule  slips  and  cost 
increases.  The  government  program  manager  should  require  integrated  CAS  plans 
as  part  of  the  prime  and  subcontractor  production  plans.  These  plans  should 
be  an  evaluation  item  for  production  readiness  reviews. 

Management  Reporting  and  Tracking.  Adequate  techniques  exist  in  industry  to 
provide  management  with  the  information  needed  to  implement  prompt  and 
effective  decisions  during  the  transition  phase.  Effective  management 
tracking  and  reporting  include  product  quality,  workmanship,  manpower 
build-up,  build-to-cost  objectives,  and  post  delivery  performance.  DoD 
publications  should  describe  the  effective  tools  that  industry  has  developed 
to  provide  senior  management  with  in-depth  program  visibility. 

Risk  Management.  Risk  management  is  a  significant  factor  in  successfully  ac¬ 
complishing  program  objectives  and  delivering  a  quality  product  on  time  and  at 
a  reasonable  cost.  Both  industry  and  government  personnel  should  be  aware  of 
the  importance  of  the  techniques  for  risk  identification  and  management. 

Facilities  and  Investment  Panel 


Industrial  Modernization  Incentives.  Government  and  industry  should  cooperate 
to  assure  a  modernized  Defense  Industrial  Base  and  mutually  share  the  risks 
associated  with  achieving  this  objective.  DoD  should  continue  to  support  the 
Industrial  Moderni2ation  Incentive  Program  (IMIP)  and  give  this  important  ini¬ 
tiative  the  widest  possible  dissemination  at  all  levels  within  Government  and 
industry. 

System  Acquisition  Procedures.  DoD  should  put  strong  emphasis  on  the  need  for 
production  planning  early  in  the  acquisition  process  and  the  involvement  of 
manufacturing  engineers  during  the  development  phase.  Industry  and  the  Govern¬ 
ment  should  recognize  the  importance  of  streamlining  proposal  requirements 
through  more  effective  use  of  Draft  RFPs.  The  application  of  existing  source 
selection  procedures  should  be  improved  by  emphasizing  the  need  for:  more 
realistic  technical  and  schedule  requirements;  additional  consideration  to 
cost  credibility  and  risk  assessment;  greater  weight  on  the  contractor's  past 
performance  and  capital  investment  programs;  and  de-emphasizing  the  importance 
of  the  lowest  proposed  cost. 

Productivity  and  Design/Manufacturing  Interaction.  DoD  should  encourage  and 
motivate  industry  to  adopt  and  use  totally  integrated  CAD/ CAM  systems. 

Data  Base  Technology.  The  key  to  a  totally  integrated  CAD/CAM  system  was 
found  to  be  the  data  base  technology  that  could  tie  together  all  elements  of 
the  design  and  manufacturing  processes  including  the  stand-alone  automated 
modules  that  may  already  exist.  Enormous  costs  are  involved  in  developing 
the  necessary  data  base  technology  and  parallel  development  efforts  within  the 
private  sector  will  be  wasteful.  Based  on  this  assessment,  the  panel 
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identified  a  national  need  for  a  standarized  approach  to  the  required  data 
base  technology  and  has  recommended  that  DoD  and  the  National  Bureau  of 
Standards  jointly  provide  a  focal  point  for  development  of  data  base 
management  system  specifications. 

Corporate  Strategic  Planning.  DoD  should  develop  a  firm,  long-term  manufac¬ 
turing  strategy  for  its  own  facilities  using  the  systems  engineering  approach. 
It  should  then  encourage  the  industrial  base  to  use  this  model  for  developing 
their  own  long  range  corporate  manufacturing  strategies. 

Human  Resources.  DoD  and  the  industrial  base  managers  should  work  with  local 
education/training  institutions  to  establish  appropriate  CAD/CAM  courses.  And 
employees  should  be  encouraged  to  attend  these  courses. 

Management  Panel 

Management  Principles.  The  panel  Identified  five  fundamental  principles  to 
enhance  the  total  process  of  development  and  production.  In  summary,  these 
are: 

o  Transition  to  production  must  be  viewed  as  an  integral  part  of  the 
Full  Scale  Development  phase. 

o  A  decision  to  proceed  with  a  program  at  Milestone  II  must  represent  a 
commitment  to  produce  and  to  deploy. 

o  An  agreed  upon  "Acquisition  Strategy"  must  exist  at  Milestone  II. 

o  The  effectiveness  of  program  managers  and  program  management  must  be 
Improved. 

o  Greater  flexibility  in  the  movement  of  funds  between  and  among 
accounts  Is  urgently  needed. 

Additional  Observations.  The  Panel  also  presented  observations  on  various 
management  topics,  including  military  specifications,  affordability,  cost 
estimating  and  data  requirements. 


Recommendations .  The  Panel  submitted  four  recommendations  related  to 
Management  Principles  and  four  recommendations  pertaining  to  their  Additional 
Observations. 


SECTION  1 


INTRODUCTION 


OBJECTIVE 

The  objective  of  this  report  is  to  provide  recommendations  that  will  lower  the 
risk  of  transitioning  weapon  systems  from  design  and  development  into  produc¬ 
tion. 

BACKGROUND 

The  success  of  our  defense  systems  acquisition  programs  over  the  past  twenty- 
five  years  has  ranged  from  exceptionally  high  to  embarrassingly  low.  Conse¬ 
quently,  there  is  a  dramatic  and  accelerating  interest  in  upgrading  the  over¬ 
all  effectiveness  of  weapon  systems,  both  those  currently  operational  and 
those  programmed  for  ultimate  delivery  to  the  services.  The  length  of  the 
acquisition  cycle,  the  unsatisfactory  levels  of  effectiveness,  and  budget 
pressures  to  reduce  the  life  cycle  costs  have  been  major  reasons  underlying 
this  renewed  concern  for  delivering  reliable  and  maintainable  weapon  systems. 
However,  in  spite  of  this  interest  and  concern,  the  problems  persist. 

A  major  cause  of  these  problems  is  the  failure  to  make  the  proper  transition 
from  design  and  development  into  production.  Recent  experience  has  shown  that 
the  application  of  proper  disciplines  and  controls  throughout  the  development 
phase  results  in  quality  products  delivered  to  the  operating  forces.  There¬ 
fore,  this  Task  Force  was  established  in  June  1982  to  examine  methods  for 
improving  the  development  process  as  well  as  the  transition  to  production. 

APPROACH 

Terms  of  Reference. 

This  Task  Force  was  formed  under  the  auspices  of  the  Defense  Science  Board  to 
review  the  process  of  transitioning  from  development  to  production.  The 
formal  terms  of  reference  are  included  in  Appendix  A  and  are  summarized  as 
follows: 

o  Examine  ways  and  methods  which  will  more  clearly  define  and 
accelerate  the  transition  from  development  into  production. 

o  Direct  the  inquiry  toward  both  the  producing  industry  and  the 
administering  government. 

o  Recommend  the  disciplines  and  controls  applicable  to  design, 
test  and  production  which  will  result  in  the  timely  delivery  of 
quality  products  to  the  operating  forces. 


In  its  deliberations  the  Task  Force  addressed  the  12  areas  affecting  the 
transition  problem  identified  in  the  Terms  of  Reference  as  well  as  several 
additional  issues  that  were  considered  relevant  to  the  objectives  of  the  study. 

Task  Force  Membership 

The  Task  Force  was  composed  of  top  level  managers  from  both  industry  and 
government.  The  membership  included  engineering,  production  and  program 
management  personnel  with  extensive  experience  in  the  development  and 
production  of  defense  systems  and  equipment.  The  team  members  applied  their 
individual,  corporate  and  government  experience  and  resources  in  clarifying 
the  issues  and  developing  possible  solutions  in  the  form  of  recommendations 
for  acquisition  decision  makers.  A  complete  list  of  the  membership  is 
included  in  Appendix  6. 

Five  Panels  Established 

The  Task  Force  divided  itself  into  five  working  panels  and  assigned  specific 
Issues  for  detailed  analysis.  These  panels  covered  the  following  areas: 

Design,  Test,  Production,  Facilities  and  Investment,  and  Management.  Each 
panel  was  chaired  by  a  Task  Force  member  and  each  member  generally  served  on 
more  than  one  panel.  Membership  on  the  Individual  panels  also  included 
additional  personnel  who  were  selected  for  their  particular  expertise  in  the 
study  area.  A  complete  list  of  the  panel  members  is  provided  in  Appendix  C. 
Coordination  meetings  of  both  the  individual  panels  and  the  full  Task  Force 
were  held  during  the  study  period.  Data  and  additional  contributions  to  the 
panel  study  efforts  were  provided  by  other  members  of  industry  and  the 
government  where  needed. 

\ 

Focus  of  Study  Effort 

The  Initial  Intention  was  to  focus  solely  on  fundamental  principles  and 
guidelines  which  were  consistent  with  established  DoD  policy  and  which  could 
be  put  into  practice  or  emphasized  without  the  need  to  change  existing 
policies.  However,  as  the  study  progressed,  the  Task  Force  realized  that  some 
of  the  findings  and  recommendations  would  involve  policy  changes  and  that 
others  would  impact  funding.  The  report  clearly  indicates  where  these  changes 
are  needed. 


Panel  Reports 

Upon  completion  of  the  work  performed  by  each  panel,  a  report  was  submitted  to 
the  Task  Force  Chairman.  Figure  1-1  shows  the  structure  of  each  report  and 
the  relation  to  the  initial  Issues  cited  in  the  Terms  of  Reference.  As  the 
study  progressed,  additional  relevant  Issues  were  Identified  and  have  been 
reported  on  by  the  panels  as  indicated  by  the  thirty-seven  topic  areas  listed 
across  the  top  of  the  matrix  (Figure  1-1).  Itshould  also  be  noted  that 
several  topic  areas  reported  on  by  Individual  panels  have  relevance  beyond  the 
implied  scope  of  the  individual  panel.  For  example,  the  issue  concerning 
subcontractor  and  vendor  management  that  was  addressed  by  the  Production 
Disciplines  Panel  is  equally  relevant  to  des  gn  and  test. 


TASK  FORCE  PANEL  REPORTS 


Structure  Related  to  Terms  of  Reference 


Upon  completion  of  the  panel  reports,  the  Chairman  of  the  Task  Force  convened 
a  special  working  committee  to  consolidate,  coordinate  and  integrate  the 
findings,  conclusions  and  recommendations  of  the  individual  panels  and  to 
prepare  the  Task  Force  report.  This  report  includes: 

o  An  Executive  Summary  that  highlights  the  results  as  reported  by  each 
paneTT 

o  A  digest  of  the  reports  submitted  by  the  panels.  This  digest  is  to  a 
level  of  detail  sufficient  to  convey  the  substance  of  each  conclusion 
and  recommendation  put  forth  by  the  panels  (refer  to  Sections  2 
through  6). 

o  A  complete  copy  of  each  panel  report  (provided  under  separate  cover), 
and 

o  A  separate  Transition  Document  which  has  been  derived  from  the  panel 
reports.  This  document  is  intended  to  serve  as  a  practical,  easy  to 
use  guide  for  identifying  and  controlling  risk  on  acquisition 
programs  and  for  improving  the  process  of  transitioning  from 
development  to  production.  (Refer  to  Appendix  D). 

o  Recommendations  for  continued  OSD/DSB  action  which  are  summarized  in 
Appendix  E. 


SECTION  2 


DESIGN  PANEL  RESULTS 


The  Design  Panel  addressed  five  principal  issues.  These  are: 

o  Disciplines  for  the  development  cycle, 

o  Figures  of  merit  for  allocated  design  time  and  cost, 

o  Key  predictors/indicators  of  potential  design  and  manufacturing 
trouble, 

o  Methods  of  displaying  design  confidence,  and 
o  Design  review  process. 

A  digest  of  the  Panel's  results  is  provided  in  this  section.  Each  principal 
issue  is  discussed  in  terms  of  background  information,  conclusions  and 
recommendations.  The  complete  Design  Panel  Report  (attached  to  this  report 
under  separate  cover)  contains  a  full  discussion  of  the  panel's  work  and 
support  for  their  recommendations. 


DISCIPLINES  FOR  THE  DEVELOPMENT  CYCLE 
Issue 

Why  do  we  have  so  much  trouble  with  the  design  effort,  and  how  can  the  process 
be  Improved? 

Background 

The  successful  accomplishment  of  a  development  program  depends  on  design 
requirements  that  are  clearly  specified  and  uniformly  understood  by  the 
procuring  agency  and  the  contractor.  The  translation  of  these  requirements 
Into  a  successful  design  calls  for  a  properly  scheduled,  adequately  funded, 
and  well-managed  program  that  is  staffed  with  an  appropriate  mix  of  designers 
possessing  the  necessary  skills  and  qualifications.  In  addition  to  their 
basic  background,  the  design  staff  must  be  properly  oriented  to  the  specific 
requirements  of  the  program  to  which  they  are  assigned,  and  they  must  be 
provided  with  the  necessary  tools. 

Guidelines  to  aid  the  design  engineer  in  his  task  do  exist.  These  guidelines 
Include  requirements  documents  issued  by  the  government  as  well  as 
contractor-generated  direction  such  as:  requirements  allocation,  design 
guidelines,  parts  control  policy,  training  programs,  etc.  These  guidelines 
are  not,  however,  consistent  throughout  industry,  nor  are  they  uniformly 
Implemented  for  all  programs.  Both  government  and  industry  agree  on 


the  importance  of  the  design  effort  and  the  necessity  to  apply  certain  disci¬ 
plines  in  this  process.  It  is  generally  acknowledged  that  systematic  imple¬ 
mentation  of  proven  design  principles  and  practices  can  lead  to  significant 
advances  In  equipment  reliability  without  excessive  added  cost.  The  conclus¬ 
ions  reached  by  the  Design  Panel  on  this  issue  emphasize  the  need  for  a  more 
disciplined  application  of  the  policies,  procedures  and  techniques  that  are 
already  established  and  generally  known  throughout  government  and  industry. 

Conclusions 

o  All  quantitative  and  qualitative  system  requirements  that  affect  the  de¬ 
sign  must  be  completely  defined,  communicated,  and  agreed  to  by  the  govern 
ment  and  the  contractor.  This  communication  begins  with  the  request  for 
proposal  (RFP)  and  Is  refined  during  contract  definitization.  Producibil- 
ity  should  be  emphasize  early  in  the  design. 

o  Contractors’  development  programs  must  be  staffed  with  a  proper  mix  of 
technical  and  management  personnel  with  the  necessary  level  of  education 
and  experience.  At  the  initiation  of  development  programs,  specific 
Indoctrination  on  program  objectives  should  be  given  to  all  levels  to 
ensure  uniform  understanding. 

o  Contractor  management  Is  responsible  for  the  design  team  having  all  the 
proper  tools,  including  a  we 11 -structured  design  program,  design  guide¬ 
lines,  computer  facllites,  and  related  data  base. 

o  Schedules  for  advanced  development  and  FSO  programs  must  be  realistic  in 
allowing  adequate  time  for  engineering  design,  analysis,  tooling,  plan¬ 
ning,  fabrication,  assembly  of  test  articles,  and  development  testing. 
Checklists  should  be  developed  and  used  to  verify  accomplishment  of  these 
tasks. 

o  Contractor  management  Is  responsible  for  ensuring  that  suppliers  and  sub¬ 
contractors  have  complete  and  definitive  design  requirements,  proper  cap¬ 
abilities  and  tools,  and  realistic  schedules. 

Recommendations 


The  Department  of  Defense  and  the  Services  should  assure  that  new-program  RFPs 
communicate  all  program  requirements.  Proposal  reviews  should  specifically 
assess  the  thoroughness  and  credibility  of  the  contractors'  responses  to  en¬ 
sure  that  all  requirements  have  been  understood  and  addressed.  The  government 
must  ensure  that  the  deflnitlzed  contract  formalizes  all  requirements  mutually 
agreed  to  by  the  government  and  the  contractor.  In  addition,  the  program  man¬ 
ager  has  a  major  influence  on  program  success,  and  as  such,  his  selection  must 
be  made  with  great  care. 
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FIGURES  OF  MERIT  FOR  ALLOCATED  DESIGN  TIME  AND  COST 


Issue 

How  can  we  ensure  that  design  time  and  cost  are  properly  estimated  and  phased? 

Background 

The  issue  suggests  the  need  for  better  techniques  to  estimate  and  allocate 
design  time  and  cost  on  development  programs.  Figures  of  merit  derived  from 
the  actual  time  and  cost  used  on  successful  programs  should  facilitate  more 
objective  and  confident  initial  planning  on  future  programs.  Additionally, 
more  confident  planning  would  help  assure  adequate  and  timely  funding,  and  the 
anticipated  results  would  be  fewer  cost  overruns  and  schedule  slippages. 

Conclusions 


o  Funding  profile  -  Continuity  of  funding  and  front-end  loading  of  funding 
were  found  to  be  significant  factors  in  determining  successful  programs. 

o  Budget  ratios  -  Figures  of  merit  for  allocation  of  budget  were  success¬ 
fully  developed  by  the  panel  to  the  first  level  of  the  WBS.  Futher  study 
and  analysis  will  be  required  to  develop  more  detailed  indicators. 

o  Schedule  -  Some  meaningful  schedule  figures  of  merit  were  developed  for 
flight  vehicles  and  their  major  components.  Factors,  such  as  system 
complexity,  contractor  capabilities  and  national  urgency,  peculiar  to 
individual  programs  were  found  to  affect  schedule  figures  of  merit. 

o  Hardware  cost  ratio  -  Insufficient  data  was  generated  to  formulate 
conclusions  in  this  area. 

Recommendation 


Either  the  Defense  Systems  Management  College  or  a  recognized,  industry- 
oriented  consulting  firm  should  be  tasked  to  (1)  review  the  data  and  findings 
of  the  Design  Panel  report;  (2)  plan  and  conduct  appropriate  research  to 
expand  the  figures  of  merit  to  the  system/subsystem  cost  level;  (3)  test  the 
application  of  these  figures  of  merit  on  trial  projects;  (4)  revise  the 
Program  Manager's  Training  Course  accordingly;  (5)  revise  and  maintain  these 
guidelines;  (6)  periodically  distribute  updates  to  these  guidelines  to 
Department  of  Defense  and  Industry  program  managers;  and  (7)  develop  a 
questionnaire  for  soliciting  opinions  and  data  from  key,  senior- level 
management  personnel. 


KEY  PREDICTORS/INDICATORS  OF  POTENTIAL  DESIGN  AND  MANUFACTURING  TROUBLE 
Issue 


How  can  we  better  achieve  timely  recognition  of  developing  program  deficien' 
cles  using  available  indicators? 


Background 


Early  detection  of  program  probiems  necessary  so  that  effective  corrective 
action  can  be  taken  before  the  problem  grows  to  a  size  that  significantly  im¬ 
pacts  or  jeoparizes  the  program.  Indicators  are,  in  effect,  the  sensors  that 
measure  progress,  indicate  current  position  relative  to  planned  position,  and 
alert  the  experienced  program  manager  to  take  timely  corrective  action.  The 
indicators  used  and  found  effective  by  the  companies  represented  on  the  panel 
are  shown  in  Figure  2-1.  These  indicators  are  discussed  in  the  Panel  report, 
and  highlighted  below. 

Conclusions 


o  Design  Problem  Indicators  -  Earliest  indicators  of  potential  design  prob¬ 
lems  are  changes  in  design  requirements  or  their  allocations.  Other  indi¬ 
cators  include  difficulties  in  negotiations  with  major  equipment  suppliers 
responding  to  procurement  specifications  or  wide  variations  in  suppliers' 
quoted  prices.  Means  for  identifying  and  correcting  design  problems  can 
be  accomplished  through  technical  performance  measurement,  test  analyze 
and  fix  (TAAF)  reliability  growth  tracking,  and  corrective  action  systems. 

o  Manufacturing  Problem  Indicators  -  The  best  indicators  of  manufacturing 
problems  generally  are  those  normalized  to  some  standard  type  of  baseline 
rather  than  absolute  numbers.  Indicators  of  such  problems  may  be  poor 
yields  detected  by  manufacturing  screens,  high  rework  levels,  and  parts 
supply  shortages. 

o  Indicators  Applicable  to  Both  Design  and  Manufacturing  -  Some  indicators 
include  work  force  composition and  stability,  and  cost  and  schedule  indi¬ 
cators. 

Recommendations 


Management  requirements  for  new  government  programs,  as  defined  in  the  request 
for  proposal  and  later  in  contractual  documentation,  should  require  the 
formal  implementation  of  a  management  system  to  identify  and  track  critical 
parameters  from  both  a  technical  and  a  program  management  viewpoint.  Proposal 
evaluators  should  assess  the  contractor's  responsiveness  to  this  requirement. 
Some  specific  indicators  that  may  be  useful  on  individual  programs  are 
discussed  in  the  Design  Panel  Report. 


METHODS  OF  DISPLAYING  DESIGN  CONFIDENCE 
Issue 


How  can  we  measure  design  confidence? 


DESIGN  TROUBLE  INDICATORS 


MANUFACTURING  TROUBLE  INDICATORS 


SCHEDULE  VARIANCE/MILESTONE 
SLIPPAGE 

COST  VARIANCE 

DESIGN  REVIEW  FINDINGS 

LATE  CDRL  ITEMS 

SUPPLIER/SUBCONTRACTOR 

PERFORMANCE 

BROKEN  TECHNICAL  PERFORMANCE 
MEASUREMENT  THRESHOLD 

FAILURE  TO  MAINTAIN  TAAF  SLOPE 

ECP  RATE 

HAYWIRES  PER  UNIT 
CORRECTIVE-ACTION  ACTIVITY 
ITEMS  IN  FAILURE  ANALYSIS 
CHANGE  TRAFFIC 


SCHEDULE  VARIANCE/MILESTONE  SLIPPAGE 
COST  VARIANCE 

WORK  FORCE  TURNOVER/ATTRITION 

ALTERNATE  METHODS  OF  FABRICATION 

PARTS  SHORTAGES 

SCRAP  RATES 

REWORK  RATES 

YIELD  RATE  OF  SCREENS 

TEST  EQUIPMENT  LATENESS 

ASSEMBLY  NONCONFORMANCES 

ASSEMBLY  MAN-HOUR  DEVIATIONS 
FROM  LEARNING  CURVE 

BENCH-TEST  TIME/ANOMALIES 

EIDP  VARIABILITY 

ITEMS  IN  FAILURE  ANALYSIS 

CORRECTIVE-ACTION  ACTIVITY 


FIGURE  2-1:  DESIGN  AND  MANUFACTURING  TROUBLE  INDICATORS 

% 
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Background 


The  development  process  begins  with  the  translation  of  design  requirements 
into  a  conceptual  design;  it  ends  with  a  documented  definition  of 
producible  hardware.  During  the  evolution  of  a  design,  confidence  is  achieved 
by  progressive  evaluation,  analysis,  review,  and  test.  Techniques  used  to 
monitor  the  design  process  for  identification  and  correction  of  problems  also 
build  design  confidence. 

Design  requirements  are  established  to  ensure  the  resulting  equipment  will 
perform  as  required  at  the  levels  of  reliability  and  maintainability 
specified.  Initially,  the  design  requirements  must  be  properly  defined  in  the 
contractual  specifications  for  the  program.  Design  margin  policies  can  then 
be  established  at  the  outset  of  the  program  and  applied  consistently. 

The  challenge  to  building  design  confidence  during  a  development  program  is 
one  of  assuring  that  the  significant  characteristics  of  a  design  have  been 
identified  early,  that  the  specified  margins  have  been  achieved,  and  that  the 
design  is  in  fact  ready  for  production.  Current  government  contracts  have 
been  Inconsistent  in  the  definition  of  required  margins  and  have  therefore  not 
provided  a  uniform  basis  for  assuring  design  confidence  prior  to  production. 
Without  this  basis,  deficiencies  often  are  not  discovered  until  after 
equipment  is  in  service,  at  which  time  the  corrective  action  to  recover 
intended  performance  is  either  too  late  or  too  costly  to  be  of  practical 
value.  The  ability  to  assess  design  capability  during  development  is  an 
essential  part  of  the  determination  of  design  confidence,  and  this  confidence 
Is  assured  by  verifying  that  specified  levels  of  design  margins  have  been 
achieved. 

Conclusions 

o  Successful  programs  force  production  design  maturity  to  occur  during 
development  so  that  the  planned  product  can  be  built  and  fully  evaluated 
before  development  is  over.  This  approach  enables  confidence  to  be 
established  in  the  design  and  the  resulting  product.  The  use  of  design 
margins  Is  also  central  to  achieving  design  confidence. 

o  A  good  way  to  gain  confidence  In  the  design  as  it  progresses  from  system 
synthesis  through  development  testing  is  through  the  use  of  an  effective 
technical  performance  measurement  system  that  enforces  self-assessment  of 
the  design  status  against  all  Important  requirements  and  objectives  on  a 
continuing  basis. 

o  Development  should  be  authorized  with  production  in  mind  from  the  outset 
In  recognition  of  the  fact  that  continuity  is  not  only  beneficial  for 
minimizing  cost  and  schedule  variances  but  necessary  to  a  successful 
transition  from  development  Into  production.  Initial  "pilot"  or 
"low-rate"  production  concurrent  with  the  later  stages  of  development  can 
be  beneficial  In  building  design  confidence  if  appropriately  used. 


Recommendations 


DoD  documents  should  be  developed  to  promulgate  the  concept  of  design  margins 
for  technical  parameters  (e.g.  margins  for  structural  integrity).  Recent 
military  programs  have  addressed  component  derating  for  electrical  parameters 
such  as  voltage  and  power  dissipation.  In  the  F/A-18  program,  the  added 
dimension  of  "design-to"  requirements  for  reliability,  which  exceeded  the 
specification  requirements,  was  introduced  as  a  means  of  providing  adequate 
margins.  A  military  standard  or  series  of  standards  to  formalize  these 
concepts  and  to  provide  a  reference  for  future  contractual  requirements  should 
be  developed. 

Technical  Performance  Measurement  (TPM)  is  currently  defined  in  M1L-STD-499. 
Procuring  services  should  develop  requirement  guidelines  for  program  sponsors 
that  require  the  contractual  implementation  of  a  TPM  system  as  a  part  of  full 
scale  development  programs. 


DESI6N  REVIEW  PROCESS 
Issues 

Why  Is  the  design  review  process  important  and  what  can  be  done  to  improve  the 
process? 

Background 

Any  contractor  involved  in  any  recent  Department  of  Defense  program  will 
immediately  acknowledge  the  importance  of  well-structured  design  reviews. 

Design  reviews  accomplish  two  main  purposes:  (1)  they  bring  to  bear  additional 
knowledge  to  the  design  process  to  augment  the  basic  program  design  and 
analytical  activity,  and  (2)  they  provide  a  formalized  means  of  verifying  the 
satisfactory  accomplishment  of  specified  design  and  analytical  tasks  as  a  basis 
for  approval  to  proceed  with  the  next  step  in  the  design  process. 

Design  reviews  fall  into  two  general  categories:  (1)  internal  reviews  in  which 
the  contractor  conducts  a  technical  review  of  his  own  design  and  (2)  external 
(or  formal)  design  reviews  conducted  jointly  with  the  government  and  the 
contractor.  The  level  and  intensity  of  the  design  review  process  must  be 
tailored  to  each  program  and  must  be  appropriate  to  the  complexity  of  the 
design  task,  level  of  development  risk,  and  criticality  of  the  design 
requirements.  Internal  reviews  are  typically  scheduled  by  the  contractor  to 
precede  the  formal  design  review  required  by  the  contract.  External  reviews 
with  the  customer  are  needed  to  obtain  continuing  design  approval  from 
government  technical  specialists. 

At  least  three  government  documents  exist  that  prescribe  the  proper  conduct  of 
design  reviews,  but  many  of  the  current  DoD  programs  do  not  invoke  these  docu¬ 
ments  as  a  contractual  requirement.  Those  that  do  cite  the  requirement  have 
not  always  properly  implemented  the  design  review  process.  Where  government 
contracts  for  defense  hardware  require  the  conduct  of  design  reviews,  the 
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main  problem  appears  to  have  been  a  lack  of  specific  direction  and  discipline 
in  the  design  review  requirement.  This  has  resulted  in  an  unstructured  review 
process  that  has  failed  to  accomplish  the  purposes  of  the  process.  The  atti¬ 
tude  of  the  procuring  service  toward  design  reviews  contributes  to  this  prob¬ 
lem  by  fostering  the  perception  that  a  design  review  is  a  mini  program  review 
to  be  conducted  for  the  purpose  of  familiarizing  people  with  an  overview  of 
hardware  design. 

Conclusions 

o  Design  reviews  should  be  a  contractual  requirement  in  any  new  hardware  or 
software  development  program.  A  design  review  plan  must  be  developed  by 
the  contractor  and  approved  by  the  government  program  manager.  This  plan 
must  define  the  design  review  policy,  schedule,  procedure,  and  partici¬ 
pants.  It  should  provide  for  both  external  design  reviews  and  internal 
design  reviews/inspections. 

o  Design  reviews  should  be  systematized  by  using  checklists  consistent  with 
design  review  guidelines  established  at  the  beginning  of  the  program. 
Internal  review  must  verify  conformance  to  the  design  review  guidelines. 
Customer  reviews  should  address  the  exceptions. 

o  Participants  in  government  design  reviews  should  be  selected  to  provide 
experience  and  expertise  to  the  review  process.  "Observers"  are  not  a 
proper  part  of  a  design  review.  It  Is  important  to  limit  the  size  of  the 
design  review  team.  Some  people  interviewed  indicated  that  10  partici¬ 
pants  is  a  workable  upper  limit.  Large  designs  should  be  subdivided  into 
workable  small  reviews. 

o  The  design  review  process  should  be  expanded  to  Include  review  of  major 
assembly  tooling.  The  requirement  for  this  activity  should  be  included  in 
the  design  review  plan  for  new  hardware  programs. 

o  The  results  of  computer-aided  design  and  analysis  should  be  used,  when 
possible,  to  support  the  design  review  process.  Examples  of  computer- 
aided  results  are  the  output  information  generated  via  computer  simulation 
or  modeling  of  a  software  system. 

o  Action  items  from  design  reviews  must  be  documented  with  assignments  of 
responsibility  for  action  and  schedules  for  completion.  Status  of  action 
items  should  be  addressed  at  the  next  scheduled  review.  The  generation 
and  follow-up  of  action  items  are  crucial  to  the  success  of  the  project. 

Recommendations 

Both  the  Air  Force  and  the  Army  currently  have  either  MIl-STD  documents  or 
design  review  guidelines  for  use  on  their  programs.  The  Navy  should  issue  a 
similar  document  with  contents  tailored  to  its  specific  needs. 

A  policy  document  should  be  issued  requiring  a  formal  design  review  plan  for 
each  major  procurement  program.  This  plan  should  be  approved  by  the  govern¬ 
ment  program  manager  for  compliance  with  policy.  Flexibility  should  be  given 
the  program  manager  to  determine  the  peculiar  requirements  of  his  program. 
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TEST  PANEL  RESULTS 


The  analyses  conducted  by  the  Test  Panel  resulted  in  the  identification  of 
eight  issues.  These  issues  deal  with: 

o  Integrated  Testing 

o  Test  Environments 

o  Reliability  Development  Testing 

o  Reliability  Demonstration  Testing 

o  Application  of  TAAF  during  Transition 

o  Initial  Operational  Test  and  Evaluation 

o  Military  Participation  in  FSED  Tests 

o  Field  Operational  Performance  Feedback 

Each  of  these  issues  is  discussed  in  this  section  in  terms  of  background, 
conclusions  and  recommendations.  The  relation  of  these  issues  to  the  areas  of 
study  identified  in  the  Terms  of  Reference  is  shown  in  Figure  1-1.  The  full 
report  of  the  Test  Panel  is  provided  as  an  enclosure  to  this  Task  Force  Report 
(under  separate  cover). 


INTEGRATION  OF  TESTS 


Issue 


How  does  integrated  test  planning  of  subsystems  and  systems  bear  on  the 
transition  from  development  to  production?  Why  is  it  important,  and  what  can 
be  done  to  optimize  the  efficiency  of  the  total  test  plan? 

Background 

Adequate  testing  is  essential  from  the  development  phase  of  a  contract  through 
initial  production  in  order  to  assure  that  the  product  meets  its  performance 
and  reliability  requirements.  Contracts  typically  call  for  tests  to  be 
performed  as  part  of  the  developmental  process.  These  tests  often  do  not 
correlate  well  with  each  other,  and  they  may  not  be  sufficient  to  ensure 
product  integrity.  Conversely,  testing  often  occurs  in  subsystems  far  in 
excess  of  common  sense.  Many  of  the  test  scenarios  are  developed  independent 
of  specific  contract  requirements,  occur  in  different  design  groups  in  the 
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contractor's  facility,  and  are  not  necessarily  coordinated  at  the  program 
level.  This  causes  redundant  testing  and  degrades  the  efficiency  of  the  test 
process. 

Conclusion 


An  integrated  test  plan  should  be  developed  early  in  the  program  to  ensure 
maximum  efficiency  in  testing.  This  plan  should  include  all  subsystem  and 
system  testing,  minimizing  redundant  tests  and  maximizing  equipment  utilization. 

Recomnendation 


The  services  should  require  an  integrated  test  plan  to  be  generated  by 
contractors  in  concert  with  the  guidelines  stated  in  the  conclusion  (above). 


TEST  ENVIRONMENTS 


Issues 


How  do  test  environments  affect  the  test  program,  and  do  they  have  an  effect 
on  the  transition  from  development  to  production?  Can  they  be  optimized  to 
reduce  schedule  times  and  reduce  costs? 

Background 

Numerous  environmental  tests,  including  vibration,  shock,  temperature, 
humidity,  and  altitude,  are  run  during  the  developmental  phases  of  a 
contract.  They  are  run  at  both  subsystem  and  system  levels.  The 
environmental  levels  are  based  on  the  top  system  level  operational 
requirements.  However,  these  requirements  may  not  be  applicable  to  all  levels 
of  the  system.  In  all  cases  the  test  requirements  established  by  the  contract 
specification  must  be  met  by  the  contractor. 

Conclusion 


One  specific  set  of  operational  environments  should  be  established  for  the 
system,  and  these  In  turn  should  be  allocated  to  each  end  item  of  equipment. 

Recommendation 


The  contractor  should  establish  subsystem  environments  in  accordance  with  the 
following  guideline: 

I 

i 

"One  specific  set  of  operational  environments  shall  be 
established  for  the  system,  and  these  In  turn  shall  be 
allocated  to  each  end  item  of  equipment.  These  environments 
shall  be  the  minimum  used  and  shall  be  tailored  to  the  expected 
environment  for  each  end  Item  during  all  environmental  tests. 

This  does  not  preclude  the  use  of  a  more  severe  environment  to 
provide  a  margin  for  actual  operation  or  to  provide  failure 
data. " 
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RELIABILITY  DEVELOPMENT  TESTS 


Issues 

How  do  reliability  development  tests  bear  on  the  transition  from  development 
to  production?  Why  are  they  important?  What  can  be  done  to  optimize  the  cost 
of  these  tests? 

Background 

Reliability  development  tests  are  conducted  under  controlled  conditions  using 
simulated  mission  environments  to  determine  design  and  manufacturing 
weaknesses.  The  tests  emphasize  reliability  growth  rather  than  a  numerical 
measurement.  Many  past  development  contracts  have  not  given  proper  emphasis 
to  reliability  development  testing,  and  test  requirements  often  have  been 
applied  arbitrarily  to  system  elements.  Both  situations  lead  to  difficulties 
in  transition  to  production. 

Conclusions 


Reliability  development  testing  should  be  integrated  with  other  development 
tests  to  minimize  resource  costs  (test  facilities,  test  hardware,  personnel) 
and  the  impact  on  program  schedule. 

Selection  of  subsystems  to  undergo  reliability  development  tests  should  be 
based  on  the  subsystem's  impact  on  the  reliability  of  the  overall  system. 

Subsystem  reliability  development  testing  should  be  terminated  when  the 
expected  reduction  In  failure  rate  becomes  small  with  respect  to  the  subsystem 
requirements. 

Recommendations 


MIL-STD-785B  should  be  revised  to  include  the  guidelines  expressed  in  the 
conclusions  (above).  Specifically,  this  includes  the  criteria  for  the 
selection  of  subsystems  and  the  termination  of  reliability  development 
testing.  Additionally,  although  MIL-STD-785B  identifies  integrated  testing, 
additional  emphasis  and  flexibility  should  be  added.  For  example,  an 
integrated  laboratory  development  test  plan  should  be  an  acceptable  alternate 
to  the  separate  reliability  test  plans  specified  in  the  Standard. 

This  revision  should  be  initiated  with  a  DoD  policy  statement  to  emphasize  the 
point  that  reliability  development  (TAAF)  tests  are  primarily  contractor  tests 
intended  to  support  the  development  of  a  reliable  production  design. 

RELIABILITY  DEMONSTRATION  TESTS 

Issues 


How  do  reliability  demonstration  tests  bear  on  the  transition  from  development 
to  production?  Are  they  important?  Are  they  cost  effective? 
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Background 

Reliability  demonstration  tests  are  conducted  to  determine,  with  a  specified 
confidence  level,  whether  the  MTBF  requirements  have  been  achieved.  The 
demonstration  tests  are  normally  performed  in  accordance  with  MIL-STD-781  and 
include  extended  test  time  under  a  specific  mission  profile.  If  required, 
they  are  performed  late  in  the  development  phase;  and  concurrent  Test,  Analyze, 
and  Fix  (TAAF)  activities  are  prohibited  due  to  the  controlled  structure  of 
the  demonstration.  Unlike  reliability  development  tests,  these  tests  are  not 
designed  to  provide  reliability  growth  for  the  transition  from  development  to 
production. 

Conclusion 


Emphasis  should  be  placed  on  reliability  development  (TAAF)  and  other 
development  testing  in  lieu  of  reliability  demonstration  testing. 

Recommendations 


MIL-STD-785B  and  MIL-STD-781C  should  be  revised  to  Incorporate  the  guideline 
Identified  in  the  conclusion  (above)  and  delete  contradictory  guidelines. 
Specific  methodologies  or  guidelines  should  be  added  to  MIL-STD-785  to  define 
Integrated  test  and  assessment  techniques.  Both  the  MIL-STD-785B  reference  to 
MIL-STD-781C  and  the  contents  of  the  latter  document  should  be  revised  to 
allow  Innovative  test  methods,  and  the  integration  of  reliability  assessment 
with  other  development  and  Initial  operational  tests. 

Since  these  revisions  may  take  considerable  preparation  and  coordination  time, 
OoD  guidelines  should  be  prepared  to  revise  current  contracting  and  program 
planning  practices  and  to  guide  the  military  standard  revisions. 


TEST,  ANALYZE  AMD  FIX 
Issues 


Should  the  test  analyze  and  fix  (TAAF)  philosophy  be  continued  on  all 
subsystem  and  system  tests  and  system  operation  during  the  transition  from 
development  to  production? 

Background 

The  TAAF  philosophy  generally  is  associated  with  tests  which  are  used  as  a 
development  tool  for  Improving  the  reliability  of  systems  and  equipment.  Such 
tests  emphasize  reliability  growth  by  using  an  iterative  test-redesign-retest 
process,  which  Identifies  corrective  action  for  equipment  design  and 
manufacturing  processes.  Reliability  growth  measuremei.t  and  tracking  can 
provide  acquisition  managers  the  best  possible  insight  into  actual  versus 
planned  progress.  It  is  becoming  top  management's  principal  tool  for 
assessing  program  readiness  for  transition  from  development  to  production. 
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Conclusion 


The  test,  analyze  and  fix  (TAAF)  philosophy  should  be  continued  on  all 
subsystem  and  system  tests  and  system  operation  during  the  transition  from 
development  to  production.  This  philosophy  should  be  coupled  with  an 
effective  closed  loop  failure  analysis  and  corrective  action  system. 

Recommendations 

Specific  OoD  guidelines  should  be  prepared  to  define  the  application  of  the 
TAAF  philosophy  during  system  and  subsystem  level  tests  and  to  provide  the 
methodology  for  reliability  growth  tracking.  Guidelines  should  emphasize  the 
need  for  a  closed  loop  failure  analysis  and  corrective  action  program  and  for 
rigorous  configuration  management  to  control  the  modification  of  system 
elements  during  the  transition  period. 


INITIAL  OPERATIONAL  TEST  AND  EVALUATION 
Issues 


During  development,  how  does  the  government  gain  confidence  that  risks  are 
being  brought  under  control,  that  the  schedule  is  on  track,  that  performance 
and  reliability  thresholds  are  being  met,  and  that  system  specifications  are 
still  appropriate  to  the  operational  need  and  threat? 

Background 

The  government's  basic  acquisition  process  is  still  founded  on  the  "fly  before 
buy"  principles  of  the  Fitzhugh  Blue  Ribbon  Panel  report  of  1970.  The  basic 
DoD  acquisition  process  makes  use  of  "milestone"  decision  points  in  each 
program  (Milestone  I  -  concept  selection  and  entry  into  demonstration  and 
validation;  Milestone  II  -  commence  FSED;  and  Milestone  III  -  complete  FSED 
and  commence  production).  At  each  milestone  DoD  decision-makers  review  T&E 
data  and  make  decisions  (proceed,  cancel,  restructure,  etc.).  The 
government's  goals  in  structuring  programs  are  to  preserve  the  options  of 
decision-makers  at  each  milestone  (i.e.,  to  avoid  precommitments)  and  to 
ensure  that  government  T&E  data  (in  addition  to  contractors'  T&E  data)  are 
available  to  decision-makers  at  each  milestone. 

Conclusion 


IOT&E  by  the  Service's  independent  test  agency  should  be  conducted  in  discrete 
phases  during  development  and  provide  data  to  support  the  decision  making 
process. 


Recommendations 


Industry  must  be  made  more  aware  of  OoD's  basic  milestone  oriented  acquisition 
policy,  as  set  forth  in  DoO  Directive  5000.1,  and  of  DoD's  fundamental  T&E 
policy,  as  set  forth  in  DoD  Instruction  5000.3.  And  DoD  Directive  5000.1  and 
DoD  Instructions  5000.2  and  5000.3  should  be  modified  to  speak  of  a 
transition-to-production  phase,  bounded  by  milestones,  rather  than  a  single 
point  at  which  development  ends  and  production  begins. 

MILITARY  PARTICIPATION  IN  FSED  TESTS 


Issues 

Does  military  service  participation  in  the  contractor's  full-scale  engineering 
development  (FSED)  phase  benefit  the  transition  of  a  weapon  system  from  devel¬ 
opment  into  production?  If  so,  how  much  participation  is  optimal?  What  are 
the  risks  involved  to  the  military  service  and  to  the  contractor? 


Development  test  and  evaluation  (DT&E)  by  the  military  services  is  normally 
conducted  as  dedicated  tests  in  discrete  periods  during  FSED.  This  requires 
the  contractor  to  allocate  test  assets  for  varying  lengths  of  time,  a  practice 
which  adversely  affects  the  continuity  of  the  contractor's  effort. 

Heretofore,  both  the  military  and  the  contractors  have  been  reluctant  to 
propose  combined  contractor-military  DT&E  in  the  FSED  phase.  The  military 
fears  loss  of  their  dedicated  DT&E  phases,  and  the  contractor  is  hesitant  to 
allow  service  test  personnel,  whom  they  view  primarily  as  evaluators/critics, 
to  use  a  weapon  system  that  is  still  early  in  its  development  stage. 

Conclusions 

Military  service  participation  (by  the  DT&E  organization)  in  contractor  FSED 
Increases  the  efficiency  of  the  testing  conducted.  Until  more  data  becomes 
available  a  tentative  goal  of  10%  Is  a  reasonable  target  for  the  amount  of 
test  effort  performed  by  the  services. 

Recommendations 

Amend  DoD  guidance  regarding  test  and  evaluation  to  encourage  agreements 
between  the  contractor  and  the  military  services  regarding  participation  in 
DT&E  during  FSED.  0T&E  should  remain  Independent  and  dedicated.  In  addition, 
action  should  be  taken  to  widely  promulgate  among  defense  contractors  the 
savings  in  cost  and  time  available  through  service  participation  in  the 
contractor  FSED  testing,  as  well  as  the  enhanced  probability  of  transitioning 
into  production  a  system  more  representative  of  what  the  service  needs. 
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FIELD  OPERATIONAL  PERFORMANCE  FEEDBACK  DURING  THE  EARLY  MANUFACTURING  PROCESS 


Issue 


During  the  initial  phase  of  service  operation  of  a  new  weapon  system,  on-site 
engineering  teams  are  often  used  to  assist  in  the  transition  to  service  use 
and  to  provide  early  feedback  of  field  problems.  Are  these  teams  an  asset  in 
the  transition  process?  If  so,  what  types  of  problems  are  identified,  and 
should  this  practice,  currently  exercised  on  an  ad  hoc  basis  in  some  cases,  be 
established  as  a  standard  policy? 

Background 

MIL-STD-785B,  "Reliability  Program  for  System  and  Equipment  Development  and 
Production",  reflects  requirements  for  establishing  a  Failure  Reporting 
Analysis  and  Corrective  Action  System  and  a  Failure  Review  Board.  These  two 
tasks  constitute  the  key  elements  of  an  effective  reliability  growth  program. 
However,  these  requirements  address  implementation  at  the  contractor's  facility 
only  and  do  not  include  provisions  for  service  use  at  remote  sites. 

In  view  of  this  situation,  a  survey  was  conducted  by  the  panel  to  inquire  into 
both  the  use  and  the  value  of  on-site  engineering  teams  during  initial  service 
operation.  The  survey  resulted  in  eleven  responses  from  major  aerospace  firms, 
electronics  manufacturers,  and  DoD  services. 

Conclusion 


Major  weapon  system  contracts  should  include  provisions  for  an  on-site 
engineering  team  to  observe  Initial  operation,  assist  in  identifying 
reliability  and  maintainability  problems,  provide  early  feedback,  and  assist 
in  data  collection. 

Recommendations 


DoD  5000.40  and  other  appropriate  policy  documents  should  be  amended  to 
reflect  the  guideline  stated  above  as  a  conclusion.  Include  provisions  in 
MIL-STD-785  for  Implementing  Tasks  104  (Failure  Reporting,  Analysis,  and 
Corrective  Action  System)  and  105  (Failure  Review  Board)  in  the  operational 
environment  during  the  transition  to  service  use. 


SECTION  4 


PRODUCTION  PANEL  RESULTS 


The  Production  Panel  addressed  the  following  issues: 
o  Vendor  Impact  on  Quality,  Cost  and  Schedule 
o  Workmanship  Defects  -  Cause  and  Relation 

o  Part  Quality  Confidence  in  Manufacturing 

o  Transition  Recipe 

o  Cooperative  Participation-Government/Industry 

o  Depot  Organic  Support 

o  Corrective  Action  Systems  (CAS) 

o  Management  Reporting  and  Tracking  Techniques 

o  Ri sk  Management 

The  following  discussion  provides  a  digest  of  the  Panel’s  analysis  of  each 
issue  and  includes  background  information,  conclusions,  and  recommendations. 
The  complete  Production  Disciplines  Panel  Report  is  forwarded  as  an  enclosure 
to  this  report. 


VEICOR  IMPACT  OH  QUALITY,  COST  AND  SCHEDULE 
Issue 

What  should  be  done  to  favorably  affect  the  vendor's  impact  on  quality,  cost 
and  schedule? 

Background 

The  percentage  of  subcontracted  work  on  major  weapon  systems  has  grown, 
reaching  as  much  as  80%  in  a  few  instances.  Hence  the  reliance  on 
subcontractors,  and  on  the  skills  of  prime  contractors  to  manage  their 
subcontractors  and  suppliers,  has  increased.  An  informal  poll  of  ten  prime 
contractors  resulted  In  statements  that  nearly  half  their  programs  were  in 
trouble  with  schedule  or  cost  because  of  major  problems  with  subcontractors. 
Clearly,  the  effective  management  of  subcontractors  needs  more  emphasis. 


Conclusions 


o  RFPs  for  prime  contracts  should  require  responses  from  bidders  with 
equitable  emphasis  on  subcontractor  management  planning  versus  in-house 
management. 

o  The  responsibility  for  managing  subcontractors  remains  with  the  prime 

contractor.  However,  the  increased  amount  of  subcontracting  may  have  left 
some  primes  with  ineffective  subcontractor  management  systems. 

o  Government  source  selection  criteria  should  give  appropriate  weighting  to 
subcontractor  management  planning;  and  prime  contractors  should  thoroughly 
evaluate  potential  subcontractor  capabilities  before  making  their 
selections. 

o  The  prime  should  keep  subcontractor  management  plans  current. 

o  Subcontractor  status  should  be  reported  against  management  plans  during 
routine  prime-customer  program  reviews.  Since  subcontractor  contact  with 
government  managers  should  be  controlled  by  the  prime,  the  prime  has  a 
large  responsibility  for  objective  and  fair  reporting. 

o  Primes  should  minimize  adversary  relationships  with  subcontractors. 
Effective  communications  from  primes  who  genuinely  want  their 
subcontractors  to  be  successful  is  an  important  factor  for  success. 

Recommendation 


Government  acquisition  planning  for  weapon  systems  should  incorporate  the 
features  listed  above  under  the  ''conclusions".  In  addition,  government 
personnel  should  recognize  the  potential  Impact  that  subcontractors  have  on 
the  success  of  major  programs  and  not  only  evaluate  the  amount  of  preplanning 
given  by  the  prime  bidders  during  the  selection  process  but  also  ensure  the 
thoroughness  of  implementation  after  contract  award. 


WORKMANSHIP  DEFECTS  -  CAUSE  AND  RELATION 
Issue 


How  can  early  identification  of  defects  and  related  causes  result  in  guidance 
for  action  during  the  development  process  to  eliminate  defects  in  the 
production  phase. 

Background 

Programs  have  gone  into  production  only  to  be  plagued  with  workmanship  defects. 
This  causes  high  rework  and  scrap  rates,  and  in  turn  results  in  missed 
schedule  and  cost  goals.  Management's  typical  response  has  been  to  throw 
resources  at  the  problem  rather  than  to  eliminate  the  cause  of  the  problem. 
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Conclusions 


o  Workmanship  defects  are  caused  by  problems  in  the  initial  system  design 
and/or  the  design  of  the  associated  production  processes.  Planning  and 
preparation  associated  with  the  production  process  may  have  inherent 
faults.  These  include  not  providing  appropriate  tools  or  equipment  to 
operators  and  not  providing  for  proper  training  of  the  production  workers 
to  assure  the  skills  necessary  for  the  job. 

o  Wien  faced  with  poor  workmanship  and  excessive  rework,  successful  actions 
have  included  communication  with  workers  (such  as  quality  circles),  being 
receptive  to  change,  audits  by  external  consultants,  special  learning 
centers  for  workers,  willingness  to  learn  from  other  companies,  and 
investment  in  enabling  tools  and  process  aids. 

o  The  general  principles  involved  in  finding  the  causes  and  correcting 
workmanship  defects  are  well  known  among  behavioral  scientists. 

Recommendations 


Innovative  techniques  are  a  means  to  cope  with  poor  workmanship  and  excessive 
rework.  A  positive  contribution  can  be  made  by  respected  officials  giving 
public  recognition  to  individuals  and  corporations  for  employing  innovative 
techniques  in  solving  these  problems. 

ESTABLISHING  PART  QUALITY  CONFIDENCE  IN  MANUFACTURING 

Issues 

Four  issues  were  related  to  improving  the  confidence  in  parts  furnished  by 
suppliers.  These  issues  dealt  with: 

o  Cost-effectiveness  of  parts  inspection  (receiving  versus  source 
inspection). 

o  Impact  of  defect  threshold  limits  for  incoming  inspection. 

o  Effective  ingredients  for  contractual  leverage  to  be  applied  by  parts 

users. 

o  Improvements  related  to  joint  government-industry  action. 

Background 

If  parts  suppliers  rigorously  tested  all  of  their  products  or  if  there  was 
effective  discipline  in  the  Military  Standard  parts  program  (Qualified 
Products  List),  then  Ideally  there  should  be  no  need  for  retesting  parts  prior 
to  use.  However,  industry  experience  regarding  the  quality  of  supplier  parts 
has  been  disappointing.  The  consequence  in  many  cases  has  been  substandard 
parts.  Panel  data  indicate  that  component  failures  are  the  cause  of  the 
highest  percentage  of  problems  at  module  and  system  level  tests.  Such  parts 
failures  cause  Interrupted  factory  flows,  rework,  workarounds,  escapes  of 
marginal  equipment  to  the  field,  and  schedule  delays. 
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Conclusions 

o  Most  contractors  prefer  heavy  emphasis  on  the  elimination  of  defects  early 
in  the  manufacturing  process. 

o  Receiving  inspection  is  almost  always  .more  cost  effective  than  source 

inspection.  Considerations  which  may  favor  source  inspection  include  cost 
of  test  equipment,  the  source  inspector's  ability  to  detect  out-of¬ 
tolerance  conditions,  and  the  need  for  in-process  inspection  for  defects 
not  readily  detectable  on  the  assembled  product. 

o  A  key  ingredient  for  maintaining  part  quality  is  timely  corrective  action 
through  rapid  feedback  of  screening  results  to  suppliers. 

o  Contractors  should  be  given  the  flexibility  to  determine  and  adjust  their 
own  defect  threshold  limits  In  receiving  inspection.  Depending  on  the 
commodity,  100%  rescreening  in  some  cases  can  be  cost-effective. 

o  Contractual  leverage  for  assuring  parts  quality  include  competition, 
quantity  buys,  "pool"  buying  with  dedicated  production  lines,  letters  of 
intent,  buyer  testing  and  feedback,  and  supplier  ratings. 

o  The  Qualified  Products  List  (QPL)  program  is  a  valid  concept,  however,  due 
to  lax  and  incomplete  government  surveillance,  it  does  not  currently 
assure  that  supplier  parts  meet  the  specified  quality  standards.  In 
addition,  screening  requirements  should  be  developed  to  supplement 
existing  specifications. 

Recommendations 

o  Government  program  managers  should  require  that  the  contractor  identify  in 
Production  Plans  the  choice  of  incoming  inspection  techniques.  Inspection 
plans  should  document  the  rationale  and  trade-offs  in  determining  how  each 
commodity  Is  to  be  Inspected. 

o  End  item  customers  should  be  encouraged  to  control  reliability  and  quality 
at  the  end  product  level  and  to  leave  flexibility  at  the  Intermediate 
steps . 

o  Discussion  of  defect  threshold  limits  and  contractual  leverage  contained 
in  the  Panel  Report  should  be  published  and  disseminated.  The  government 
should  re-establish  the  QPL  program  as  an  effective  program,  and  the 
Institute  of  Environmental  Sciences  (IES)  should  complete  its  current  work 
on  a  uniform  set  of  rescreening  requirements. 

RECIPE  FOR  ENTERING  PRODUCTION 

Issue 

What  are  the  principal  Ingredients  in  the  recipe  for  transitioning  from 

development  into  production? 
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Background 


The  Panel  observed  that  there  is  very  little  in  any  formal  government  policy 
or  contractual  requirement  which  lays  the  ground-work,  or  even  recognizes  the 
need  for  taking  specific  actions  during  the  development  phase  to  assure  a 
smooth  and  successful  transition  to  production.  Contractors,  because  of  bad 
experience  in  making  the  transition,  have  started  to  develop  and  enforce 
internal  requirements  to  assure  that  a  successful  production  program  results 
from  actions  taken  early  in  advanced  development  and  throughout  the 
engineering  development  phases.  While  it  is  always  within  the  contractor's 
prerogative  to  take  whatever  actions  are  necessary,  some  are  costly  and  time 
consuming,  and  most  could  be  better  effected  if  their  value  were  recognized  by 
the  customer  and  if  they  were  joint  contractor-customer  actions. 

Conclusions 

o  The  goverment  program  manager  should  require  and  fund  the  contractor's 
production  plan,  prepared  initially  no  later  than  the  start  of  engineering 
development  and  continually  updated  until  rate  production  is  achieved. 

o  Development  contracts  should  require  and  fund  a  formal  design-to-unit 
production  cost  program,  producibility  phase,  and  a  manufacturing 
involvement  plan. 

o  A  formal  production  readiness  review  should  be  jointly  conducted  by  the 
customer  and  the  contractor  prior  to  the  start  of  initial  production. 

o  The  government  should  fund  proof-of-design  and  proof-of-manufacturing 
models. 

Recommendation 

Each  of  the  conclusions  noted  above  should  be  included  as  specific  line  items 
In  development  and  intlal  production  contracts. 

COOPERATIVE  PARTICIPATION  BY  G0VERM1ENT/INDUSTRY 


How  can  a  climate  of  cooperative  participation  be  created  between 
representatives  of  industry  and  government  to  affect  a  smooth  transition? 

Background 

Uhtil  rate  production  has  been  achieved  and  at  least  a  year's  quantity  of 
hardware  has  been  delivered,  contractors  need  flexibility  to  optimize  the 
producibility  of  the  design,  solve  basic  design  errors  and  omissions,  and 
optimize  the  manufacturing  process.  Current  government  policies  with  respect 
to  control  of  the  technical  data  package,  granting  of  local  material  review 
board  (MRS)  authority,  classification  of  changes,  and  identification  of 
problems  and  solutions  have  had  an  adverse  affect  on  the  contractor's 
flexibility.  Extended  cycle  times  for  change  approvals  can  cause  schedule 


delays  and  potential  retrofits.  In  addition  contractors  have  a  difficult  time 
in  convincing  customers  that  "find,  analyze  and  fix"  at  the  earliest  stages  is 
to  the  customer's  long  term  benefit.  The  identification  of  problem  areas  by 
contractors  sometimes  leads  to  a  government  attitude  of  punitive  reaction  and 
results  in  adversary  relationships. 

Conclusions 


o  Government  control  of  the  technical  data  package  should  be  deferred  until 
the  conclusion  of  first  year's  production  deliveries  or  IOC,  whichever 
occurs  later. 

o  If  multiple  sourcing  of  prime  production  programs  is  contemplated,  it 
should  not  occur  until  rate  production  has  been  achieved  and  the  maturity 
of  the  technical  data  package  demonstrated. 

o  Once  the  government  takes  control  of  the  technical  data  package,  the 
contractor  should  be  granted  local  MRB  authority,  and  on-site  authority 
should  be  maintained  to  determine  classification  of  changes. 

o  A  more  constructive  attitude  should  be  adopted  by  the  government  on  the 
early  identification  and  elimination  of  design/production  deficiencies. 
Qualified  customer  personnel  should  be  assigned  on  site  to  assist  the 
contractor  in  timely  "find,  analyze,  and  fix"  actions.  Schedule  delays 
resulting  from  such  actions  should  not  result  in  penalties. 

Recommendation 


Conclusions  are  consistent  with  established  policy  and  should  be  implemented 
under  existing  authority  as  appropriate  on  individual  programs. 


DEPOT  ORGANIC  SUPPORT 
Issue 

Would  the  delay  of  implementing  depot  organic  support  help  facilitate  the 
trans  ion  process? 

Background 

There  is  considerable  evidence  that  support  postures  have  been  measurably 
improved  by  early  emphasis  on  planning  and  integration  of  all  support 
disciplines  into  the  weapon  system  acquisition  process.  However,  there  is 
also  evidence  to  suggest  that  logistic  constraints,  when  imposed  too  early  in 
the  process,  have  resulted  in  limiting  the  rapid  and  even  the  ultimate 
maturation  of  weapon  systems  to  their  Tull  potential.  The  consequence  is 
higher  life  cycle  costs  and  lower  weapon  s, -stem  capability  for  the  dollar. 

The  current  practice  of  the  earliest  possible  transition  to  organic  depot 
(in-house)  capability  was  observed  by  the  Panel  to  be  largely  wasteful.  An 
example  of  an  alternate  approach  to  organic  support  is  provided  by  the  way 


commercial  inertial  navigation  systems  are  handled.  Under  this  arrangement 
contractors  have  a  stake  in  maintenance  costs  as  well  as  production  costs,  and 
they  can  act  quickly  because  they  have  a  higher  degree  of  autonomy  with  regard 
to  changes  than  with  most  military  programs  where  logistics  concerns  cause  an 
early  constraint  on  the  ease  of  making  changes.  The  Panel  gathered  data  from 
the  various  Services  which  futher  supports  the  finding  that  contractor  depot 
support  is  more  cost  effective  and  responsive  than  organic  support. 

Conclusions 


o  Contractors  should  maintain  control  of  the  configuration  at  least  2-3 

years  beyond  deployment  to  allow  for  maturation.  Organic  depot  operations 
should  be  deferred  during  this  period  and  contractors  should  be  given 
total  analysis  and  repair  responsibility  until  configuration  control  is 
transferred  to  the  customer. 

o  Contractor  technical  representatives  should  be  assigned  in  the  field  for 
dynamic  feedback  of  critical  perf ormanc e/problem  information  and  to  aid  in 
training  and  orientation  of  military  personnel  at  the  organizational  and 
intermediate  levels. 

o  A  product  improvement  warranty  should  be  intitiated  with  contractors.  It 
should  provide  sufficent  incentive/penalty  provisions  regarding 
performance  and  logistic  support  measurements  to  ensure  full  contractor 
motivation  within  the  early  years  of  field  introduction. 

Recommendation 


The  above  conclusions  should  be  implemented  under  existing  DoD  authority. *  In 
many  cases,  some  of  the  commands  are  already  operating  under  these  guidelines; 
in  the  others,  it  will  only  require  a  change  of  policy  within  the  commands. 


CORRECTIVE  ACTION  SYSTEMS 
Issue 


What  are  the  essential  elements  of  effective  corrective  action  systems? 

Background 

Currently,  two  OoO  documents  are  invoked  on  most  significant  development 
contracts  to  cover  the  subject  of  corrective  action  systems (CAS).  The  intent 
may  be  paraphrased  as  follows:  "Establish  a  system  which,  with  suitable  data 
analysis  and  investigation  as  to  problem  cause,  is  capable  of  implementing 
changes  in  a  timely  manner.  Such  changes  shall  be  effective  in  precluding 
recurrence  of  the  problem."  Since  any  effective  system  typically  requires 
considerable  participation  from  virtually  all  functional  organizations,  this 
can  represent  a  considerable  challenge.  When  prime  contractors  pass  these 
requirements  to  subcontractors  there  are  additional  interface  difficulties. 

In  addition,  under  pressure  to  close  the  non-recurring  charges,  contractor 
production  support  budgets  are  usually  insufficient  to  support  the  continued 
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levels  of  effort  for  expected  CAS  activity  throughout  transition.  In  present 
defense  weapon  system  contracting  procedures,  CAS  is  not  generally  recognized 
as  an  essential  program  line  item,  and  its  legitimacy  is  suspect.  The 
consequence  of  not  sustaining  an  effective  CAS  throughout  transition  can  be  to 
delay  the  true  "ready-for-production"  date,  resulting  in  downstream  schedule 
slips  and  cost  increases. 

Conclusions 

o  Essential  technical  aspects  of  an  effective  CAS  include  data  collection, 
data  analysis,  investigation,  design  fix,  effectivity  of  the  fix, 
effectiveness  of  the  fix,  timeliness  and  management  support. 

o  CAS,  properly  supported  and  funded,  can  be  very  effective  in  identifying 
and  resolving  problems  during  the  transition  from  development  into 
production. 

o  Because  of  the  concept  of  detailed  data  transfer  between  prime  and 

subcontractors,  primes  sometimes  overmanage  their  subs.  This  results  in 
documentation  burdens,  prohibitive  costs  and  interface  difficulties.  As 
an  alternative,  primes  should  employ  more  effective  management  techniques 
using  program  reviews. 

o  Contractor  management  is  not  always  fully  aware  that,  where  corrective 
action  is  needed,  an  earlier  identification  and  correction  of  the  problem 
will  provide  a  higher  return  on  investment  of  company  assets. 

o  Funding  still  represents  the  life  blood  of  tasks  and,  unless  funding 
visibility  is  improved,  good  intentions  will  be  insufficient  to  achieve 
the  CAS  goal.  Separately  defined  funding  for  CAS  should  be  evaluated  in 
proposals. 

Recommendations 


The  DoO  management  should  communicate  its  commitment  to  CAS  through  the 
contracting  agencies  of  the  government.  The  government  program  manager  should 
require  integrated  CAS  plans  as  part  of  the  prime  and  subcontractor  production 
plans.  The  proposed  CAS  system  should  be  further  evaluated  at  production 
readiness  reviews. 


MANAGEMENT  REPORTING  AND  TRACKING  TECHNIQUES 
Issue 

What  techniques  should  be  employed  to  provide  management  visibility  for  prompt 
and  effective  decisions  with  respect  to  schedule,  technical,  and  financial 
problems? 

Background 


Segments  of  the  industrial  community  have  been  effective  in  developing 
management  reporting  and  tracking  techniques  which  permit  responsible  program 
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and  senior  level  management  to  gain  early  visibility  of  potential  problem 
areas.  Near  real  time  reporting  techniques  afford  management  the  opportunity 
to  direct  company  resources  to  a  timely  solution  of  transition  bottlenecks. 
This  assures  meeting  transition  cost,  schedule  and  technical  goals. 

Conclusions 


o  Adequate  techniques  already  exist  in  industry  to  provide  management  with 
the  visibility  required  to  implement  prompt  and  effective  decisions  during 
the  transition  phase.  Reporting  techniques  coupled  with  periodic 
management  reviews  are  sufficient  to  provide  early  insight  into  potential 
problems  which  could  impact  the  transition  from  a  schedule,  technical,  or 
financial  point  of  view.  Attentive  management  should  be  able  to  cope  with 
any  areas  of  concern  to  effect  an  orderly  transition. 

o  Effective  management  tracking  and  reporting  includes  product  quality, 
workmanship,  component  quality,  manpower  build-up  ,  build  to  cost 
objectives,  and  post-delivery  performance.  All  of  these  factors  are 
contributors  in  one  form  or  another  to  the  total  acquisition  cost. 

Recommendations 


An  appropriate  DoD  publication  should  be  prepared  to  describe  the  effective 
tools  that  industry  has  developed  already  to  provide  senior  management  with 
in-depth  visibility. 


RISK  MANAGEMENT 
Issue 


What  are  the  elements  of  aggressive  risk  management,  and  how  can  they  be  used 
effectively  in  assuring  good  quality,  cost  and  schedule  performance? 

Background 

Each  year  Industry  spends  huge  sums  of  money  on  the  quality  of  the  products  it 
produces  as  well  as  on  programs  to  achieve  optimum  cost  and  schedule 
performance.  The  industrial  objective  is  to  deliver  a  quality  product  in  a 
timely  fashion  at  the  least  cost.  One  of  the  factors  in  successfully 
accomplishing  this  objective  is  effective  risk  management.  The  risks  vary 
considerably  from  one  product  to  another,  but  a  methodology  can  be  developed 
for  the  achievement  of  this  objective  at  minimal  cost. 

Conclusions 


Three  elements  of  risk  were  identified  as  a  framework  around  which  an 
effective  risk  management  program  can  be  structured: 

1.  Sensitivity  to  risk.  The  degree  to  which  risk  awareness  is  experienced 
determines  an  organization's  ability  to  “feel  the  risk".  The  organization 
must  be  willing  to  accept  the  fact  that  the  best  intentions  do  not  always 
work  out  and  that  the  best  technique  to  cope  with  the  future  is  to  assess 
the  probability  of  difficulty  and  plan  for  problems. 
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2. 


Identification  of  the  risks.  Experience  is  the  cornerstone  of  efforts  to 
identify  risks.  Reviewing  prior  knowledge  and  early  establishment  of 
thorough  test  programs  very  often  will  reveal  areas  of  concern  to  the 
organization.  Candidates  for  designation  as  a  risk  item  include  any  item 
which  potentially  could  jeopardize  schedule  or  cost  constraints,  items 
which  cannot  be  tested  or  inspected,  sole  source  items  or  items  with  an 
inconsistent  test/quality  history.  In  addition,  a  high  level  of  change 
activity  to  meet  performance  requirements  and  excessive  procurement  lead 
times  would  also  flag  trouble  areas. 

3.  Dealing  with  the  risks.  To  minimize  the  effect  of  identified  risks,  they 
should  first  be  prioritized  with  respect  to  criticality.  Organizational 
decision  making  can  also  be  upgraded  by  making  sure  it  becomes  more 
institutionalized  and  less  haphazard.  Continuity  of  program  personnel 
should  also  be  maintained  during  transition  in  order  to  benefit  from  past 
experience  and  to  maintain  consistency. 

Recommendations 


Industry  and  government  personnel  involved  in  the  management  of  weapon  systems 
programs  should  be  made  aware  of  the  importance  of  risk  identification  and 
management.  The  techniques  indicated  herein  and  available  from  other  sources 
should  be  used  as  appropriate  for  the  program  to  avoid  risk-related  problems. 


SECTION  5 


FACILITIES  AND  INVESTMENT  PANEL  RESULTS 


The  assignment  given  to  the  Facilities  and  Investment  Panel  was  to  focus  on 
tasks  7  and  8  in  the  Terms  of  Reference,  namely: 


o  Economics  of  design  confidence  and  of  manufacturing,  and 

o  Minimum  requirements  for  manufacturing  capital  investments. 

While  the  first  task  specifically  involved  a  financial  analysis  of  the 
benefits  of  Computer-Aided  Design  and  Computer-Aided  Manufacturing  (CAD/CAM) 
systems,  both  tasks  involved  the  overriding  issue  of  capital  investment  -  its 
formulation,  benefits  and  inhibitors.  The  Panel  quickly  reached  unanimity 
that  the  focus  of  its  attention  should  be  directed  towards  capital  investment 
as  well  as  the  "DoO  business  environment". 

However,  recognizing  the  significant  productivity  improvement  benefits  to  be 
derived  from  CAD/CAM,  this  area  was  selected  for  further  critical  review  by  a 
subcommittee  established  under  the  leadership  of  Mr.  Frederick  J.  Michel, 
Director  for  Manufacturing  Technology,  HQ  US  Army  DARCOM.  The  members  of  this 
subcommittee  were  drawn  from  the  CAD/CAM  user  community,  the  systems 
community,  the  financial  community,  the  industrial  publications  community, 
academia,  leading  experts  in  the  field  and  appropriate  government  agencies. 
Their  names  appear  in  Section  III  of  the  Panel  Report. 

The  Facilities  and  Investment  Report  reflects  the  intermingling  of  both  cost 
and  technical  considerations.  In  keeping  with  the  overall  spirit  of  the  DSB 
Task  Force  directives,  attempts  were  made  to  provide  guidelines  to  be  used  for 
policy  formulation  by  the  heads  of  each  service  of  the  Department  of  Defense 
as  well  as  guidance  that  could  be  flowed  down  to  acquisition  managers.  In 
some  instances  guidelines  were  not  practical,  but  the  issue  was  considered  of 
such  significance  to  the  Panel's  goals  and  objectives  that  a  position  paper 
was  prepared.  These  position  papers  are  identified  in  the  final  paragraph  of 
this  section  and  included  in  the  enclosed  Panel  report  (under  separate  cover). 

The  results  from  the  work  of  the  Panel  are  presented  in  six  major  areas: 

o  Industrial  modernization  Incentives 

o  System  acquisition  procedures 

o  Producitivity  and  design/manufacturing  interaction 

o  Data  base  technology 


o  Corporate  strategic  planning 

o  Human  resources 

The  following  discussion  provides  a  digest  of  the  Panel's  assessment  of  the 
problem  in  each  area  and  recommendations  for  corrective  action  as  well  as 
guidelines  for  use  in  system  acquisition  decision-making.  The  detailed 
findings  and  supporting  rationale  are  contained  in  the  Panel  Report. 


INDUSTRIAL  MODERNIZATION  INCENTIVES 


Discussion  of  Problem 

Capital  Investment  to  modernize  the  Defense  Industrial  Base  in  the  United 
States  is  inadequate  to  maintain  a  strong  defense  posture  and  a  world 
leadership  position.  Other  industrialized  nations  continue  to  modernize  at  a 
faster  rate  and,  in  some  instances,  surpass  the  United  States  in  terms  of 
manufacturing  productivity.  Modernization  means  lower  cost,  which  relates 
directly  to  lower  profit  potential.  More  lucrative  commerical  investment 
opportunities  have  lured  capital  investment  away  from  the  Defense  Industrial 
Base. 

A  particularly  good  example  of  the  application  of  a  modernization  program  is 
reflected  in  the  business  arrangement  between  the  USAF  and  Westinghouse 
Electric  Corporation  (WEC).  This  is  a  continuing  agreement  to  share  savings 
for  approved  capital  investments  that  modernize  production  and  result  in  lower 
prices  to  the  government.  The  business  deal  provides  a  mechanism  whereby  WEC 
can  propose  capital  investments  to  the  USAF  to  be  incentivized  through 
production  contracts. 

The  way  this  happens  is  through  an  unprecedented  business  arrangement  known  as 
"GET  PRICE"  (Productivity  Realized  through  Incentivizing  Contractor 
Efficiency).  This  unique  initiative  by  the  Air  Force's  Electronic  Systems 
Division  recognizes  that  contractor  operations  are  process  rather  than  product 
oriented  and  resolves  the  long  standing  problem  of  motivating  capital 
investment  and  modernization.  Although  centrally  managed,  the  modernization 
effort  is  executed  through  individual  program  offices  which  get  immediate 
benefits  through  system  price  reduction.  Through  shared  savings,  incentives 
are  provided  for  modernization.  With  multi-program  and  possible  multi-service 
applications,  this  creative  arrangement  also  enhances  Government/ industry 
mutual  trust  and  cooperation.  The  "GET  PRICE"  concept  is  further  explained  in 
the  briefing  provided  in  Appendix  B  to  the  Panel  Report. 


Recommendations 


The  government  and  industry  should  cooperate  to  mutually  share  associated 
risks  in  order  to  assure  a  modernized  Defense  Industrial  Base.  Also  the 
Department  of  Defense  should  continue  to  support  the  Industrial  Modernization 
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Incentives  Program  ( IM IP)  and  emphasize  the  importance  of  this  initiative  by 
giving  it  the  widest  possible  dissemination  at  all  levels  of  the  government 
and  industry. 


SYSTEM  ACQUISITION  PROCEDURES 
Discussion  of  Problem 

Emphasis  does  not  always  shift  from  development  to  production  as  the 
acquisition  concept  indicates.  Government  and  industry  production  management 
personnel,  working  in  a  design  engineering  and  development  oriented  community, 
have  often  been  unable  to  effect  procedures  to  assure  the  adequate 
consideration  of  producibility  as  an  integral  part  of  the  design  process. 
Production  risk  is  inadequately  assessed  and  orderly  planning  of  timely 
resolution  is  difficult.  Program  management's  preoccupation  with  system 
performance,  initial  operational  capability  schedules,  and  development  cost 
has  also  been  reflected  in  minimal  proofing  programs  for  production  processes 
and  tooling  because  these  requirements  compete  for  the  development  dollar. 
Similar  considerations,  aggravated  by  the  drive  for  cost  reduction,  have 
prevented  the  incorporation  of  essential  requirements  for  production  tasks  in 
contractual  instruments. 

Extensive  and  at  times  unnecessary  proposal  requirements  continue  to  pervade 
government  procurement  documentation,  adding  to  overall  cost  and  detracting 
from  the  data  most  necessary  for  making  decisions. 

The  source  selection  process  has,  at  times,  encouraged  program  cost  growth  due 
to  lack  of  cost  realism,  inaccurate  assessment  of  risk  or  inadequate 
assessment  of  the  given  cost  data. 

Recommendations 

DoD  should  put  special  emphasis  on  the  need  for  involving  manufacturing 
engineers  earlier  in  the  design  process  both  for  new  and  on-going  programs. 

And  further,  OoD  should  provide  a  strong  emphasis  in  its  CAD  efforts  to 
include  the  manufacturing  function  as  part  of  an  integrated  CAD  system.  All 
acquisition  managers  should  include  production  planning  as  a  specific 
requirement  in  system  development  contracts.  When  funding  cuts  occur, 
acquisition  managers  should  be  advised  to  give  production  planning  equal 
priority  with  other  considerations. 

Both  industry  and  the  government  recognize  the  importance  of  streamlining 
proposal  requirements  through  more  effective  use  df  the  Draft  Request  for 
Proposal  {DRFP).  When  appropriate,  government  and  industry  managers  should 
conduct  face-to-face  reviews  of  planned  program  requirements  and  cost  drivers 
before  or  during  the  DRFP  stage.  Government  should  use  this  technique  more 
frequently  and  Industry  should  seriously  consider  and  recommend  needed  RFP 
changes  without  fear  of  being  considered  non-responsive.  Government  program 
managers  should  heavily  weigh  industry  recommendations  for  improving 
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productivity  and  manufacturing  producibi 1 ity.  They  should  give  these 
considerations  equal  priority  with  system  performance  and  operational 
schedules. 

OoD  should  improve  the  application  of  existing  source  selection  procedures  by 
emphasizing  the  need  for: 

o  More  realistic  technical  and  schedule  requirements  in  the  RFP. 

o  Additional  consideration  to  cost  credibility  and  risk  assessment. 

o  Greater  weight  on  the  contractor's  past  performance  and  capital 
investment  programs. 

o  Re-emphasizing  the  importance  of  the  government's  most  probable  cost 
number  in  the  evaluation  process  and  de-emphasizing  the  importance  of 
the  lowest  proposed  cost. 


PRODUCTIVITY  AND  DESIGN /MANUFACTURING  INTERACTION 
Discussion  of  Problem 


In  addition  to  the  classic  problem  of  lack  of  interaction  between  design  and 
manufacturing,  there  exists  a  declining  rate  of  productivity  improvement. 
CAD/CAM  is  a  totally  integrated  manufacturing  system  composed  of  a  series  of 
Individual  modules.  These  can  be  operated  under  a  common  data  base.  When 
operating  as  stand-alone  modules,  productivity  improvements  of  20%  or  better 
have  been  experienced.  A  totally  integrated  CAD/CAM  system  can  provide  even 
greater  productivity  improvements  in  the  industrial  base,  as  well  as  a  basis 
for  overcoming  the  classic  problem  of  lack  of  design/manufacturing  interaction. 

Recommendation 


OoD  should  provide  the  necessary  encouragement  and  motivation  to  industrial 
base  plant  managers  to  adopt  and  use  totally  integrated  CAD/CAM  systems. 


DATA  BASE  TECHNOLOGY 


Discussion  of  Problem 


The  key  to  a  totally  integrated  CAD/CAM  system  is  a  data  base  management 
system  that  ties  all  elements  of  the  plant  and  the  stand-alone  modules 
together.  Many  modules  already  exist,  except  in  the  area  of  geometric 
modeling.  Data  base  technology  is  the  shortfall.  There  is  a  national  need 
for  the  development  of  data  base  technology.  Special  consideration  should  be 
given  to,  and  emphasis  placed  on,  the  development  of  geometric  modeling  and 
the  integration  of  the  geometric  and  hierachical  data  bases. 
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Recommendation 

DoD  and  the  National  Bureau  of  Standards  (NBS)  should  jointly  provide  a  focal 
point  for  the  development  of  specifications  for  a  data  base  management  system 
with  DoD  providing  the  seed  money  for  this  effort.  The  development  of  data 
base  architecture,  communications  protocols  and  reliability,  data  base 
standards,  and  software  languages  should  be  addressed  in  two  efforts:  first, 
the  development  of  the  specifications  for  a  data  base  management  system;  and 
second,  the  development  of  the  data  base  management  system  itself. 


CORPORATE  STRATEGIC  PLANNING 
Discussion  of  Problem 

Most  corporate  strategic  planning  in  the  U.S.  is  financial  or  marketing 
oriented.  Few,  if  any,  corporations  have  strategic  plans  for  manufacturing. 
Consequently,  in  transitioning  material  from  design  to  manufacturing, 
opportunities  for  achieving  substantially  lower  manufacturing  costs  are  not 
realized.  The  success  of  the  Japanese  at  implementing  manufacturing 
technology  is  due,  in  large  part,  to  their  philosophy  of  planning  for  the  long 
haul.  Their  plans  are  modularized  and  then  prioritized  on  the  basis  of  an 
estimated  Return  on  Investment  (ROI).  Also,  a  well  communicated  national 
policy  is  the  basis  for  the  corporate  strategy  of  manufacturing. 

Recowinendatlon 

DoD  should  develop  a  firm,  long-term  manufacturing  strategy  for  its  own 
facilities  using  the  systems  engineering  approach.  It  should  then  encourage 
the  Industrial  base  to  use  this  model  for  developing  their  own  long  range 
corporate  manufacturing  strategies. 


HUMAN  RESOURCES 
Discussion  of  Problem 

Management  must  understand  that  the  application  of  CAD/CAM  will  create  the 
need  for  new  job  skills  and  will  eliminate  many  current  jobs.  Each  function 
performed  in  design  and  manufacturing  organizations  will  be  impacted.  There 
is  a  need  for  formal  training  and  re-training  programs  to  create  a  work  force 
in  the  U.S.  that  is  responsive  to  current  and  future  technology  needs.  The 
lack  of  trained  personnel  is  a  major  constraint  to  implementing  technology. 

Recommendation 

DoD  and  the  industrial  base  managers  should  work  with  local  education  and 
training  Institutions  to  establish  appropriate  CAD/CAM  cojrses.  Employees 
should  be  encouraged  to  attend  these  courses.  Encouragement  should  be 
provided  by  including  in  performance  reviews,  a  measure  of  participation  in 
educational  opportunties. 


POSITIOM  PAPERS 


The  Facilities  and  Investment  Panel  addressed  ten  additional  issues  that 
required  specific  changes  in  established  policies  or  congressional  legislation 

A  position  paper  was  prepared  for  each  issue  and  is  included  in  the  Panel 
report.  A  synopsis  of  these  recommended  changes  is  provided  here. 

Recovery  of  Interest  Expense  as  a  Cost 

CAS  414  recognizes  that  when  a  contractor  invests  in  facilities  or  equipment 
to  perform  a  government  contract,  the  imputed  cost  of  the  interest  on  such 
investment  is  in  fact  a  legitimate  cost  of  performance  which  could  be 
recovered.  In  the  implementation  of  CAS  414  by  OPC  76-3,  DoD  has  negated  the 
intent  of  CAS  414. 

In  order  to  implement  CAS  414  as  intended,  the  imputed  cost  of  capital  should 
be  recognized  for  what  it  is,  an  allowable  cost,  and  such  costs  should  be 
included  in  the  cost  base  for  the  computation  of  contract  profit. 

Recovery  of  CAD/CAM  Expense  (CAS  402  and  418) 

The  purpose  of  CAS  402  "Consistency  in  Allocating  Costs  Incurred  for  the  Same 
Purpose"  is  to  preclude  the  double  counting  of  costs.  The  purpose  of  CAS  418 
"Allocation  of  Direct  and  Indirect  Cost"  is  to  distinguish  between  direct  and 
indirect  cost  and  to  assure  that  such  costs  are  consistently  classified. 
CAD/CAM  systems  development  and  operation  costs  should  be  classified  as 
indirect  costs  and  allocated  to  higher  and  broader  levels  of  the 
organization.  Such  allocation  could  be  at  a  total  Engineering  or  Production 
level  for  example,  or  even  at  an  Engineering/Production  Operations  level. 

DoD  directives  should  be  issued,  directing  DCAA  and  DCAS  to  broaden  their 
interpretations  to  allow  CAD/CAM  costs  to  be  classifed  as  indirect  and 
allocated  on  as  broad  a  basis  as  appropriate  while  still  assuring  current 
period  cost  assignment  and  allocation  to  the  organizations  utilizing  the 
CAD/CAM  system  and  receiving  benefits  from  it. 

Independent  Research  and  Development  IR&D 

Present  OoD  policies  do  not  sufficiently  stimulate  the  development  of  new 
manufacturing  technology  and  productivity  improvements  for  plant  operations. 

IR&D/B&P  ceilings  should  be  increased  so  that  ceilings  are  within  a  more 
realistic  correlation  to  actual  expenditures,  up  to  a  limit  based  on  historic 
actual  IR&D/B&P  expenditures  of  the  contractor  as  a  percentage  of  sales  for 
the  previous  five  years. 


Multiyear  Procurement  (MYP)  will  allow  for  considerable  cost  reduction 
opportunities  in  programs  if  the  right  kind  of  funding  policies  and  support 
are  provided.  Multiyear  programs  need  funding  to  support  economic  order 
quantities  (EOQ)  and  efficient  production  rates.  Funding  policies  in  DoO 
should,  as  a  matter  of  course  and  without  need  for  special  approvals,  allow 
unfunded  cancellation  ceilings  to  include  both  recurring  and  non-recurring 
costs. 

Competition 


Increased  competition  has  long  been  recognized  as  an  effective  means  of 
reducing  program  cost  and  improving  productivity.  The  present  DoD/industry 
business  environment  does  not  promote  effective  competition. 

Facilities  Capital  Employed 

There  is  an  underabsorption  of  contractors'  facilities  capital  employed  in 
performance  of  Department  of  Defense  contracts.  The  only  change  needed  is  a 
revision  to  DoD  policy  presently  stated  in  DAR  3-808.7  to  allow  Cost  of 
Facilities  Capital  Employed  for  Services  as  well  as  Research  and  Development 
contract  efforts,  or  to  require  allocation  of  Facilities  Capital  Employed  on  a 
basis  that  includes  manufacturing  activities  only,  excluding  both  Services  and 
Research  and  Development  direct  charge  amounts  from  the  allocation  base. 

Foreign  Military  Sales 

More  efficient  use  of  FMS  dollars  could  provide  enhanced  capital  formulation 
and  additional  incentive  for  capital  expenditures  for  modernization.  DoD 
should  consider  selective  changes  to  policy  so  that  all  the  restrictive 
requirements  of  DAR,  which  were  legislated  and/or  developed  for  use  on 
appropriated  funded  contracts,  are  not  made  applicable  to  the  non-appropriated 
funded  FMS  contracts. 

Technology  Transfer 

Government  sponsored  manufacturing  technology  can  greatly  enhance  contractor 
productivity  if  effective  technology  transfer  is  employed.  The  need  for 
technology  transfer  has  been  discussed  for  years,  but  we  have  not  done  a  very 
good  job  of  communicating  DoD  funded  manufacturing  technology  programs  much 
less  transferring  the  results  among  DoD  contractors.  In  addition,  we  have  to 
understand  that  one  of  the  basic  advantages  of  technology  transfer  is  improved 
productivity  by  reducing  planning  and  duplication  of  effort. 

Manufacturing  Research 

Research  in  manufacturing  technology  is  just  recently  being  recognized  with 
funding  as  a  viable  means  for  Improving  our  defense  posture.  This  area  needs 
continued  and  Increased  support.  To  support  this  expanding  need,  the  DoD 
should  director  or  encourage  the  inclusion  and  expansion  of  manufacturing 
research  (RDT8£)  funding  as  an  intregral  part  of  the  MANTECH  Program. 
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Productivity  Payback 


DoD  should  adopt  policies  and  procedures  to  enable  a  commitment  to  be  made  in 
the  first  production  contract  that  productivity  gains  developed  and 
implemented  during  performance  of  the  contract  would  be  shared  in  follow-on 
production  contracts.  To  be  an  effective  motivator,  the  contractor's  share 
must  be  significant  and  go  beyond  what  is  available  under  DAR  3-808. 8(a). 


SECTION  6 


MANAGEMENT  PANEL  RESULTS 


The  Management  Panel  focused  their  attention  in  five  major  areas.  Several  of 
these  areas  were  examined  by  other  panels  from  a  predominantly  technical 
viewpoint  and  are  discussed  in  previous  sections  of  this  report.  However,  a 
management  persp>»ctive  was  considered  important  to  fully  address  all  aspects 
of  the  issues  involved.  The  five  areas  are: 

o  Military  Specification  Problems 

o  integration  of  Design  and  Manufacturing 

o  Technical  Data  Management 

o  Entry  into  Production 

o  Program  Manager  and  Program  Management 

At  the  outset,  the  Panel  members  received  various  briefings  pertaining  to  the 
above  issues  and  related  the  substance  of  these  briefings,  in  a  qualitative 
sense  with  their  own  extensive  experience  in  their  respective  areas  of 
expertise.  A  preliminary  set  of  conclusions  and  recommendations  was  then 
drafted  which  were  generally  directed  toward  policy  and/or  procedural  changes 
needed  to  Improve  the  management  of  the  development  to  production  transition. 
Since  all  Task  Force  Panels  were  requested  to  concentrate  on  actions  which 
could  be  Implemented  with  little,  if  any,  modification  to  existing  OSD  policy, 
the  Management  Panel  re-evaluated  its  initial  results.  Whereas  compliance 
with  this  request  was  easier  to  accomplish  for  those  panels  reviewing  the  more 
technically  oriented  issues,  the  Management  Panel,  even  with  a  disciplined 
attempt  to  restrict  its  focus,  found  that  their  continuing  discussions  led 
them  back  to  five  key  "principles"  with  four  major  recommendations.  The  Panel 
also  made  several  "observations"  which  resulted  in  four  additional 
recommendations.  These  recommendations  were  considered  to  have  high  payoff 
potential,  although  requiring  some  change  in  OSD  and/or  Congressional  policy. 

Proceeding  with  this  approach,  the  panel  found  that  several  of  the  emerging 
principles,  observations  and  recommendations  overlapped  more  than  one  of  the 
five  major  areas,  which  were  described  previously  as  the  initial  assignment  of 
the  Management  Panel.  Consequently,  the  Management  Panel  Report  as  well  as 
this  section  of  the  Task  Force  Report  is  structured  not  around  the  five 
initial  task  areas,  but  around  the  five  key  principles  and  several 
observations  which  the  Management  Panel  members  agreed  were  important.  At  the 
end  of  this  section  are  the  recomnendatlons  that  resulted  from  the  panel's 
deliberation. 
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Principle  #1:  "Transition  Phase**  Versus  Production  Milestone 

All  elements  of  government  and  industry  involved  in  acquisition  must 
understand  that  "Transition  to  Production"  is  an  integral  part  of  the  Full 
Scale  Development  phase.  The  Design  to  Production  transition  is  a  process  and 
not  a  fixed  event,  beginning  not  later  than  Milestone  II. 

Background.  This  principle  reflects  the  Panel's  view  that  preparing  for 
production  has  not  received  the  necessary  emphasis  in  the  DoD  approach  to 
structuring  acquisition  programs.  The  early  phases  of  the  program  (both  pre- 
and  post -Milestone  II)  almost  totally  concentrate  on  demonstrating  whether 
system  performance  can  be  met  within  certain  economic  and  reliability 
criteria.  The  question  of  whether  the  design  can  be  produced  while  retaining 
the  system  performance  is  usually  not  answered  until  late  in  developments 

In  large  part,  DoD  policies  and  practices  over  the  last  10  to  12  years  have 
caused  the  transition  problem.  The  very  existence  of  a  "Milestone"  for 
approval  of  production  implies  a  point  in  time  where  production  actions  are 
initiated. 

In  1969-1970,  when  the  Fitzhugh  Blue  Ribbon  Panel  was  charting  a  course  away 
from  McNamara's  total  package  procurement,  and  when  "fly  before  buy"  was 
coined  to  describe  the  new  philosophy,  the  typical  program  structure  of  an 
RDT&E/  acquisition  program  was  as  shown  in  Figure  6-1. 

This  simple  graphic  -  repeated  countless  times  in  documents,  briefings, 
journals,  etc.  -  had  a  powerful  impact.  It  created  in  people's  minds  the  idea 
that  there  Is  an  Instant  In  time  In  each  program  prior  to  which  all  effort  is 
RDT&E,  and  after  which  only  production  activity  takes  place.  Unfortunately, 
this  misconception  resulted  In  many  ill-advised  rules.  "Production  funds 
cannot  be  expended  before  Milestone  III."  "RDT&E  funds  will  not  be  authorized 
after  Milestone  III."  "Test  hardware  must  be  procured  with  RDT&E  funds." 

"Test  hardware  must  be  product ion- representative."  "Production  lines  will  be 
established  using  procurement  funds."  Such  is  the  power  of  a  simple  concept, 
never  clearly  refuted,  that  even  today,  more  than  a  decade  later,  its  legacy 
ties  us  in  knots  as  we  attempt  to  transition  to  production. 

In  addition,  three  other  factors  were  Identified  by  the  Panel  that  have  tended 
to  deter  efforts  leading  to  production.  These  were: 

o  a  policy  which  clearly  discouraged  any  concurrency 

o  a  strong  set  of  advocates  for  completion  of  expanded  testing  before 
granting  approval  to  produce,  and 

o  for  a  period  of  time,  defense  guidance  which  encouraged  starting  more 
development  programs  than  could  be  carried  through  to  production. 

The  catch  phrase  was,  "R&D  for  the  shelf." 
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FIGURE  6-1:  TYPICAL  PROGRAM  STRUCTURE  —  1969-1970 


planning,  tooling  and  test  equipment  and  other  actions  leading  to  product k 
The  standard  question  was,  "How  can  you  request  procurement  funds  when  you 
not  through  development?" 


Conclusions.  The  lack  of  emphasis  on  getting  ready  for  production  is  caused 
in  part  by  the  perceptions  created  by  the  "Milestone"  approach.  In  addition, 
current  restrictions  on  the  application  of  funds  have  made  the  problem 
difficult  to  solve.  Programs  which  had  an  integrated  approach  to  development 
and  production  generally  had  successful  production  programs. 


Principle  #2:  Commitment  to  Produce  and  to  Deploy 

Milestone  II  is  the  commitment  to  produce  and  deploy.  All  organizational 
elements  of  tRe  DoO  (and  particularly  the  budget  allocators)  must  be 
instructed  to  program,  budget  and  release  funds  to  support  the  process  leading 
to  production.  At  Milestone  II  all  necessary  funds  -  RDT&E,  Production, 
Military  Construction,  etc.,  must  be  programmed  to  support  the  total  effort. 
Actual  fund  releases  for  other  than  RDT&E  will  be  event  related  (such  as  test 
events). 

Background.  This  principle  is  closely  related  to  Principle  #1  dealing  with 
the  recognition  of  transition  to  production  as  an  integral  part  of 
development.  The  significance  and  implications  of  the  decision  to  start  full 
scale  development  have  varied  through  several  versions  of  DoDD  5000.1.  Early 
versions  had  a  strong  emphasis  on  a  sequential  approach  with  little  overlap. 
Budgeting  and  programming  activities  were  (and  still  are)  structured  to 
minimize  procurement  investment  during  the  Full  Scale  Engineering  Development 
phase.  Milestone  III  became  a  formidable  barrier,  in  part,  because  there  was 
not  a  clear  commitment  to  produce  the  developed  item  at  the  time  FSED  was 
initiated. 

A  commitment  to  produce  and  to  deploy  was  found  to  be  a  necesary  and 
fundamental  step  in  the  process  of  integrating  production  activities  with  the 
design  process.  On  this  point,  a  1977  DSB  Task  Force  studied  the  factors 
contributing  to  the  lengthening  of  the  acquisition  cycle.  Among  their 
recommendations  was  one  to  revise  5000.1  to  — 

"Explicitly  state  that  approval  for  Full  Scale 
Development  includes  the  intent  to  deploy." 

There  were  other  recommendations  for  revisions  to  5000.1.  An  OSD  Task  Force 
was  established  to  rewrite  the  directive,  and  a  revision  was  published  in 
March  1980.  It  stated  that  the  Milestone  II  decision  also  means  the  Secretary 
of  Defense  intends  to  deploy  the  system.  However,  This  policy  statement  was 
short-lived.  A  recent  revision  (March  1982)  does  not  include  the  statement. 
Without  a  clear  policy,  the  funds  necessary  for  critical  production  activities 
will  not  be  provided,  and  the  current  perception  that  production  activities  do 
not  start  until  Milestone  III  will  continue.  It  must  be  emphasized  that 
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Milestone  III  is  not  the  decision  to  produce  and  deploy.  Milestone  III  is  a 
program  event  which  provides  the  opportunity  to  re-affirm  the  Milestone  II 
decision. 


The  current  version  of  this  directive,  which  reflects  the  Acquisition 
Improvement  Initiatives,  made  another  change  which  the  Panel  viewed  with  some 
concern  because  it  runs  counter  to  their  recommended  principles.  In  the 
judgment  of  the  panel,  the  start  of  full  scale  development  is  the  major 
decision  in  a  program.  It  is  a  commitment  to  develop,  produce,  and  deploy  the 
system.  " 

The  March  1982  version  of  DoDO  5000.1  appears  to  have  a  "floating"  Milestone 
II,  which  could  take  place  some  time  after  a  Service  initiates  full  scale 
development.  What  is  the  Secretary  of  Defense's  decision  point  for  program 
go-ahead?  We  can  visualize  full  scale  development  contracts  with  no  Secretary 
of  Defense  program  approval.  With  a  program  in  this  indefinite  status,  it 
seems  unlikely  that  a  Service  would  make  the  investments  for  producibi 1 ity. 
Full  scale  development  approval  and  program  go-ahead  should  be  synonymous. 

If  Milestone  II  is  a  commitment  to  produce  and  deploy  -  assuming  development 
Ts  successful  -  then  funding  for  production  activities  must  be  made  available 
at  appropriate  times  during  the  development.  "Appropriate"  is  not  easy  to 
define,  but  data  developed  supports  a  conclusion  that  preparing  for  production 
should  start  early  in  FSED. 


There  are  at  least  two  principal  accounts  that  fund  production  preparation 
efforts.  The  OSD  Comptroller's  Manual  draws  a  distinction  between  "Items 
approved  for  procurement'1  and  “Items  not  approved  for  procurement. "  In  the 
first  case,  tooling  and  test  equipment  common  to  both  the  development  and 
procurement  phases  are  to  be  financed  by  procurement  appropriations.  In  the 
second  case,  tooling  and  preliminary  production  facilities  required  to  provide 
realistic  development  hardware  are  financed  by  RDT&E  -  even  though  they  may 
later  be  used  for  production.  Clearly,  the  key  is  whether  a  decision  has  been 
made  for  procurement.  Without  such  a  decision,  the  often  beleagured  RDT&E 
account  must  finance  production  start  up  activities.  In  addition  the  RDT&E 
account  supports  producibility  studies,  design  to  cost  studies,  tooling  and 
test  equipment  planning,  manufacturing  planning,  etc. 

Informal  discussions  with  both  government  and  contractor  program  managers  were 
held.  Their  view  was  that  RDT&E  funds  programmed  for  these  purposes  were 
always  inadequate  and  that,  even  when  programmed,  they  were  "raided"  when 
development  problems  occurred.  The  Management  Panel  is  of  the  strong  opinion 
that  funds  must  be  programmed,  budgeted  and  allocated  during  the  development 
process  to  achieve  any  reasonable  chance  of  a  successful  transition  to 
production!  The  funds  can  come  from  either  an  enhanced  RDT&£  budget  or  from 
H|  procurement  funds.  Supporting  funds  such  as  MILCON  should  also  be  programmed 

The  pace  of  production  preparation  activities  will  be  set  by  progress  in  the 
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Conclusions 


The  current  version  of  OoDD  5000.1  does  not  explicitly  state  that  Milestone  II 
is  a  commitment  to  produce  and  to  deploy  and  the  "floating"  Milestone  II 
leaves  programs  in  an  indefinite  status.  Full  scale  development  approval  and 
program  go-ahead  should  be  synonymous. 

Principle  #3:  Acquisition  Strategy 

An  agreed  upon  "Acquisition  Strategy"  must  exist  at  Milestone  II.  The 
Acquisition  Strategy  identifies  the  major  decisions  which  affect  the  conduct 
and  cost  of  the  program.  It  becomes  the  "contract"  between  the  program 
manager,  his  seniors,  and  the  interfacing  and  supporting  organizations.  (It 
also  is  the  "contract"  between  the  Service  and  OSD.)  Examples  of  subjects 
addressed  are: 

o  Type  of  contracting  for  both  development  and  production, 

o  Extent  of  competition, 

o  System  integration  responsibility, 

o  Use  of  multi-year  contracts, 

o  Technical  data  package, 

o  Test  criteria. 

o  Required  resources  by  account  and  by  year. 

o  Requirements  imposed  by  supporting  organizations. 

Changes  imposed  on  the  program  manager  can  then  result  in  "contract"  changes, 
with  appropriate  cost,  schedule  and  performance  relief.  To  support  the 
strategy,  a  complete  Program  Plan  must  exist  which  defines  who  will  do  what  to 
execute  the  program. 

Background.  An  acquisiton  strategy  must  exist  at  Miletone  II  (at  the  start 
of  full  scale  development).  DoDD  5000.1  states: 

"Develop  an  acquisition  strategy  at  the  inception  of  each  major 
acquisition  that  sets  forth  the  objectives,  resources, 
management  assumptions,  extent  of  competition,  proposed 
contract  types,  and  program  structure...." 

The  most  recent  draft  of  DoDI  5000.2  (October  1982)  requires  that,  at 
Milestone  II,  a  section  of  the  Decision  Coordinating  Paper  (DCP)  address  the 
Acquisition  Strategy.  Further,  when  an  Integrated  Program  Summary  (IPS)  is 
required,  the  strategy  is  a  sub-element  of  a  procurement  section.  In  both 
cases,  it  appears  that  the  importance  of  a  well-defined,  agreed-upon  strategy 
is  not  adequately  stressed. 
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The  Panel  concluded  that  the  acquisition  strategy  should  be,  in  effect,  the 
agreement  or  contract  between  the  Program  Manager,  his  service  seniors,  and 
interfacing  and  supporting  organizations.  A  condensed  version  should  be  the 
agreement  between  the  service  and  OSD.  This  version  would  address  the  major 
management  issues  which  have  substantial  impact  on  the  conduct,  cost  and 
funding  profile  of  the  program.  These  generally  would  be  of  such  criticality 
that  they  would  not  be  delegated  to  the  Program  Manager.  He,  of  course, 
should  be  heavily  involved  in  the  development  of  recommendations. 

It  is  very  important  to  define  those  decisions  which  are  considered  the 
responsibility  of  top  management  (either  service  or  OSD).  In  general,  they 
would  be  issues  which  have  major  impact  on  funding  and  schedule.  Examples  of 
such  issues  are: 

o  Extent  of  competition  of  both  development  and  production 
o  Management  of  system  integration 

o  Production  rates  and  related  facilities 

o  Extent  of  test  program 

o  Need  for  re-procurement  -  i.e.,  extent  and  content  of  Technical  Data 
Package 

o  Type  of  contracting  by  program  phase,  including  multi-year 
considerations 

o  Degree  of  pre-planned  product  improvement  (P3I) 

o  Definition  of  high  risk  areas  that  require  parallel  or  alternate 
approaches 

q  Logistics  approach 

o  Requirements  imposed  by  supporting  organizations 

The  output  of  the  strategy  should  be  funding  profiles  by  account  and  by 
year.  These  profiles  would  support  a  complete  program  from  start  of  full 
scale  development  to  deployment. 

The  Acquisition  Strategy,  by  itself,  is  not  sufficient  for  the  conduct  of  a 
program.  It  must  be  supported  by  program  plans  which  are  the  operating 
documents  and  generally  quite  detailed.  These  plans  define  the  who,  what, 
when  and  where  of  the  program.  The  services  have  policies  and  detailed 
procedures  for  preparation  of  such  plans.  For  example,  AFSCP  800-3,  "A  Guide 
for  Program  Management,"  has  been  in  existence  for  seven  years. 

The  tough  management  job  in  DoD  is  to  prevent  the  Acquisition  Strategy  from 
becoming  the  Program  Plan.  If  the  Acquisition  Strategy  can  be  held  to  truly 
significant  issues,  it  will  be  a  major  management  tool. 


Conclusions.  The  Acquisition  Strategy  is  a  most  important  document  and  a 
clear  understanding  of  what  issues  are  to  be  addressed  in  the  strategy  is 
necessary.  The  details  of  program  management  should  be  placed  in  a  Program 
Plan. 

Principle  #4;  Program  Managers/ Program  Management 

The  effectiveness  of  program  managers  and  program  management  must  be  improved. 

Background.  The  Panel  conducted  a  survey  of  the  Services  and  contractors 
regarding  selection,  training,  career  patterns  and  length  of  assignment  of 
program  managers.  In  addition,  information  from  52  military  program  managers 
was  obtained  regarding  their  views  of  program  management.  Among  the  subjects 
covered  were  manning  levels,  matrix  management,  colocation  of  personnel,  and 
division  of  time  between  "outside"  and  "inside"  activities.  As  a  result  of 
this  information,  the  Panel  concluded  that  the  effectiveness  of  program 
managers  and  program  management  must  be  improved. 

The  survey  indicated  that  the  services  have  reasonably  good  career  development 
programs,  except  for  the  Navy,  where  a  formal  program  does  not  seem  to  exist. 
The  Air  Force  and  Navy  occasionally  use  civilian  program  managers  and  judge 
their  performance  to  be  excellent.  The  Army  has  no  plans  for  civilian  program 
managers. 

The  survey  also  indicated  that  the  average  tour  length  for  military  program 
managers  has  improved,  but  is  still  less  than  an  average  of  three  years.  By 
contrast,  industry  program  managers  have  tours  of  up  to  10  years  with  four  to 
six  years  being  quite  common.  Most  companies  have  training  programs  of  some 
sort,  but  the  approach  varies  widely.  On-the-job  training  seems  most  used. 

And  both  DoD  and  industry  consider  a  program  management  assignment  as  career 
enhancing. 

Colocation  of  functional  directors  with  the  program  manager's  office  was  found 
to  be  used  by  industry  whenever  possible.  The  lack  of  colocation  is  viewed  as 
a  serious  problem  by  military  program  managers. 

Three  other  related  issues  were  discussed  by  the  Panel.  These  issues  dealt 
with  program  managers  and  related  personnel  requirements. 

o  Program  Manager  Continuity.  DoDD  5000.23  -  "System  Acquisition 
Management  Careers"  -  was  issued  in  1974  to  establish  policies  for 
selection,  training  and  career  development  of  program  managers.  It 
is  a  good  document,  but  it  seems  to  have  been  largely  ignored. 
However,  it  does  have  one  provision  that  should  be  reconsidered.  The 
directive  states,  "Changes  of  Program  Managers,  if  necessary,  should 
normally  occur  near  major  program  milestones..."  (underlining  is  in 
the  original  document). 

One  of  the  major  conclusions  of  this  Task  Force  is  that  start-up  of 
production  Is  one  of  the  most  difficult  periods  of  systems 
•development.  If  there  is  one  time  that  continuity  of  management  is 


critical,  it  is  during  this  phase  of  early  production  and  the  related 
operational  testing.  The  Management  Panel  suggests  that  this 
provision  is  not  correct  and  should  be  deleted. 

o  Program  Office  Responsible  for  One  Program.  Several  of  the  briefings 
to  the  Panel  indicated  that  the  services  are  moving  in  the  direction 
of  assigning  more  than  one  significant  program  to  a  single  program 
office.  The  reason  given  was  usually  that  personnel  shortages  are 
forcing  such  actions.  The  contractor  members  of  the  panel  noted  that 
the  services  would  be  less  than  happy  if  the  contractors  took  such  a 
step.  Managing  a  major  program  takes  the  full  time  (or  more)  of  a 
manager.  If  the  data  received  is  correct,  we  strongly  urge  that  any 
move  in  the  direction  of  multi -programs  per  program  office  be 
carefully  reviewed. 

o  Line  Management  Control  Over  Personnel  Functions.  The  Panel  was 

briefed  on  the  Civil  Service  experiment  being  conducted  at  the  naval 
Weapons  Center,  China  Lake,  and  at  NOSC,  San  Diego.  The  thrust  of 
the  experiment  is  to  provide  local  organizations  greater  line 
management  control  over  personnel  functions.  Primarily,  it  provides 
local  management  flexibility  in  recruiting  and  pay  scales,  and  links 
pay  to  performance.  Salary  offers  to  entry  level  scientists  and 
engineers  are  flexible,  and  these  organizations  have  been  able  to 
compete  more  effectively  for  new  graduates.  The  experiment  has  been 
underway  for  several,  years  and  the  results  are  very  encouraging. 

The  Panel  was  informed  that,  because  of  the  success  to  date,  it  is 
being  considered  for  application  in  many  other  Service  technical 
organizations.  The  Management  Panel  supports  this  initiative. 

Conclusions.  The  following  conclusions  were  drawn  by  the  panel: 

o  Tour  lengths  for  military  program  managers  appear  too  short. 

u  Lack  of  colocation  of  program  office  and  functional  support  personnel 
is  a  serious  problem. 

o  There  is  an  increasing  use  of  government  civilian  program  managers, 
except  in  the  Arn\y. 

o  The  Civil  Service  experiment  being  conducted  at  several  naval 
installations  is  very  encouraging. 

Principle  #5:  Funding  Flexibility 

In  order  to  improve  the  management  of  programs,  the  Secretary  of  Defense 
urgently  needs  greater  flexibility  in  movement  of  funds  between  and  among 
accounts. 

Background.  Quite  often  toward  the  end  of  development,  the  Program  Manager 
has  inadequate  RDT&E  funds  remaining,  but  he  has  production  funds  that  he 
cannot  effectively  utilize  because  of  program  stretch-outs  or  a  startup  that 
was  slower  than  planned.  The  Secretary  of  Defense  currently  has  limited 


authority  ($750  million  in  FY  1982)  to  reprogram  funds.  The  requirement  for 
four  Congressional  committee  approvals  makes  it  unwieldy  and  time  consuming. 
Approval  of  such  transfer  action  may  take  from  six  to  eight  months. 

Acquisition  Improvement  Program  Initiative  #15  addressed  the  need  for  greater 
funding  flexibility.  In  the  1983  budget  submittal,  the  OSD  proposed  that  they 
be  allowed  to  transfer  funds  between  RDT&E  and  procurement  accounts  in  a  given 
fiscal  year  for  an  individual  weapon  system. 

The  House  Appropriations  Committee  was  less  than  enthusiastic  about  the 
request.  Their  committee  report  states: 

"Such  a  change  would  lessen  the  ability  of  the  Congress  to  maintain 
oversight,  and  would  add  to  the  already  serious  problem  of  transitioning 
prematurely  from  development  to  production.  The  Committee  is  strongly 
opposed  to  the  proposed  revision.  The  Committee  handles  reprogramming 
requests  expeditiously,  and  will  continue  to  do  so." 

The  Panel  strongly  supports  OSD's  efforts  to  obtain  greater  funding 
flexibility. 

Conclusion.  In  order  to  improve  the  management  of  programs,  the  Secretary  of 
Defense  urgently  needs  greater  flexibility  in  movement  of  funds  between  and 
among  accounts.  In  spite  of  congressional  opposition,  efforts  to  secure 
greater  flexibility  should  be  continued. 


ADDITIONAL  OBSERVATIONS 
Military  Specifications 

Background.  One  of  the  tasks  assigned  to  the  Panel  concerned  the  adequacy  of 
Mil -Specs  and  standards.  The  scope  was  expanded  by  the  Panel  to  include: 

o  "Tailoring"  to  specific  programs 

o  Are  Mil -Specs  an  asset  or  a  liability? 
o  How  to  keep  Mil-Specs  updated 

o  What  could  be  used  in  lieu  of  mil  specs? 

The  Panel  reviewed  the  DSB  study,  "Report  of  the  Task  Force  on  Specifications 
and  Standards,"  dated  January  15,  1977.  That  Task  Force  concluded  that 
specifications  and  standards  should  not  and  cannot  be  eliminated  from  the  DoD 
procurement  system.  The  Panel  found  nothing  that  would  challenge  that 
conclusion. 


After  submittal  of  the  1977  study,  DoDD  4120.3  "Defense  Standardization  and 
Specifications  Program,"  was  revised  to  reflect  many  of  the  recommendations. 
In  addition,  a  new  DoDD  4120.21  was  issued  which  provided  the  rules  for 


application  and  tailoring.  The  current  version  is  dated  November  3,  1980. 

The  directive  is  reasonably  clear,  but  implementation  has  been  difficult. 
Tailoring  is  a  tough  job  which  requires  experienced  people.  Quite  often,  in 
the  press  to  issue  an  RFP,  tailoring  falls  by  the  wayside.  Contractors 
perceive  that,  if  they  propose  tailoring,  they  will  be  considered 
non-responsive.  This  Panel  believes  that  an  adequate  framework  for  tailoring 
exists  but  with  little  motiviation  for  implementation.  Some  management 
attention  would  help. 

The  Panel  received  a  briefing  on  Mil-Prime.  This  is  a  program  at  AFSC's 
Aeronautical  Systems  Division  which  is  developing  specifications  and  standards 
stated  in  terms  of  operational  needs.  It  has  built-in  tailoring  since,  in 
effect,  the  procuring  agency  is  forced  to  fill  in  its  own  numbers.  The 
briefing  was  encouraging,  but  this  approach  is  not  used  throughout  AFSC. 

The  Panel  took  a  limited  look  at  the  problems  of  keeping  specifications  and 
standards  up-to-date.  Although  each  document  is  assigned  to  a  specific 
organization  for  keeping  it  current,  the  update  program  is  very  much  behind 
schedule.  The  two  basic  problems  are:  1)  lack  of  technical  personnel  for 
what  is  considered  a  low-priority  job,  and  2)  no  identified  funding  for  the 
effort.  The  Panel  believes  that  adequate  funding  for  this  effort  would  be  a 
useful  step.  In  addition,  automating  the  process  should  be  examined. 

Conclusions.  In  summary,  the  specific  conclusions  of  the  Panel  on  the  subject 
of  military  specifications  are: 

o  Specifications  and  standards  are  necessary  for  the  DoD  procurement 
system. 

o  "Tailoring"  is  well  defined  but  is  not  implemented  with  vigor. 

o  AFSC's  Aeronautical  System  Division  Mil-Prime  approach  appears  to 
have  value,  but  its  application  is  very  constrained.  Consideration 
should  be  given  to  expanding  the  program. 

o  The  update  program  for  standards  and  specifications  is  seriously 
behind  schedule,  primarily  because  of  no  identified  funding. 

Data  Requirements 

Background.  While  examining  the  Specifications  and  Standards  issue,  the 
Panel  received  a  briefing  from  the  USAF  Electronics  System  Division  (ESD)  on  a 
study  of  ESD  data  requirements.  The  objective  of  the  study  was  to  assess  the 
perceived  problem  that  the  government  requires  too  much  data  of  all  types  on 
acquisition  contracts  and  to  recommend  appropriate  actions.  The  Management 
Panel  also  requested  industry  views  on  data  requirements. 


The  ESD  study  team  concluded  that  the  government  did,  in  fact,  ask  for  too 
much  data  in  requests  for  proposals  and  contracts;  however,  it  was  impossible 
to  quantify  this  conclusion.  Direct  costs  for  data  ranged  from  an  estimated 


six  percent  to  twenty  percent  of  the  contracts,  and  this  does  not  include 
overhead  costs,  nor  does  it  include  the  cost  to  the  government  of  processing, 
reviewing  and  managing  the  data. 

Industry  responses  to  the  Management  Panel  requests  for  information  can  be 
summarized  as  follows: 

o  The  Cost/Schedule  Control  System  (C/SCS)  frequently  is  applied  in  a 
very  inflexible  way,  requiring  too  much  detail.  The  level  of 
reporting  should  be  raised  to  work  breakdown  structure  (WBS)  level  3, 
and  the  requirement  for  analysis  should  be  limited  to  significant 
cost  and  schedule  variances.  The  requirement  for  SARs  should  be 
waived  when  ample  evidence  exists  that  the  company  has  a  good  C/SCS 
system. 

o  The  amount  of  data  required  from  contractors  is  probably  excessive, 
but  it  is  difficult  to  quantify.  This  appears  to  be  a  problem  that 
must  be  worked  out  for  each  program.  The  problem  can  be  minimized 
only  by  much  cooperative  effort  on  the  part  of  both  the  customer  and 
the  contractor.  In  most  cases,  it  appears  that  overall  costs  could 
be  reduced  by  using  the  internal  company  reports  and  data  rather  than 
imposing  a  unique  government  requirement. 

Conclusion.  Tailoring  of  contract  data  requirements  lists  (CDRLs)  is 
difficult,  gets  lip  service,  but  little  action.  Other  DoD  organizations 
should  review  the  ESO  report  for  possible  implementation. 

Affordability 

Background.  Several  versions  of  DoD  Directive  5000.1  have  stressed 
affordability.  The  most  recent  version  (March  1982)  stresses  the  relationship 
between  the  PPBS  and  DSARC  process  and  states:  "Approval  to  proceed  into 
full-scale  development  or  into  production  shall  be  dependent  on  DoD  Component 
demonstration  that  resources  are  available  or  can  be  programmed  to  complete 
development,  to  produce  efficiently  and  to  operate  and  support  the  deployed 
system  effectively.  Funding  availability  shall  be  reaffirmed  by  the  DoD 
Component  before  proceeding  into  production  and  deployment."  Reference  is 
then  made  to  "specific  facets  of  affordability,"  which  are  in  DoDI  5000.2. 
Unfortunately,  DoDI  5000.2  does  not  say  anything  specific.  It  appears  that 
there  is  some  confusion  regarding  the  application  of  the  concept  of 
affordability. 

While  the  inputs  were  far  from  exhaustive,  the  Panel  concluded  that  there  is 
not  a  consistent  approach  to  affordability.  The  Panel  also  concluded  that  a 
new  program  which  approaches  the  full  scale  development  decision  point  should 
be  judged  against  the  total  service  program  and  the  resource  commitments  made 
by  previous  decisions..  If  a  "bow-wave"  already  exists  there  Is  little 
justification  for  a  new  program  start  unless  some  other  previously  approved 
program  can  be  cancelled  or  reduced  in  scope  (which  certainly  doesn't  enhance 
stability).  Until  there  Is  a  realistic  look  at  the  implications  of  previous 
decisions,  more  programs  will  continue  to  enter  FSED  than  can  be  afforded  in 
later  years.  This  is  a  very  serious  problem  which  seems  to  be  recognized  but 
is  not  being  resolved. 


Conclusion.  The  question  of  affordability  deserves  greater  scrutiny  at  tne 
time  that  Full  Scale  Engineering  Development  is  contemplated. 

Cost  Estimating  and  Program  Change. 

Background.  The  Management  Panel  feels  that  industry  and  government  do,  not 
only  an  acceptable,  but  normally,  a  very  excellent  job,  in  generating  cost 
estimates  for  a  program.  However,  as  the  program  is  executed,  too  many 
occasions  arise  where  changes  are  made  to  the  program  which  result  in  cost 
growth  and  the  attendant  loss  of  overall  management  credibility. 

Conclusion.  To  minimize  this  situation,  a  great  effort  must  be  exerted  at 
al 1  levels  in  Defense  to  resist  any  changes  to  a  program.  When  an  acquisition 
strategy  is  agreed  upon  and  the  program  is  being  executed,  changes  to  the 
program  must  be  exceptions  to  policy  and  not  'de-facto'  policy. 


RECOMMENDATIONS 


Four  major  recommendations  were  derived  from  the  "principles"  previously 
discussed  in  this  section: 

1.  Revise  DoDD  5000.1  and  OoDI  5000.2  to  reflect  the  policy  that 
"Transition  to  Production"  is  an  integral  part  of  the  Full  Scale 
Development  process.  Specifically,  make  the  following  changes: 

o  State  that  Milestone  II  is  the  commitment  to  produce  and  deploy. 

o  Require  that  at  Milestone  II,  al_l_  necessary  funds  are  programmed 
-  RDT&E,  Procurement,  MIL  CON,  etc. 

o  Define  what  should  be  included  in  the  Acquistion  Strategy  and 
make  it  the  "contract"  between  the  Service  and  OSD. 

o  Limit  the  Acquisition  Strategy  to  major  issues.  Require  that 
details  of  a  program  be  in  Program  Plans. 

o  Delete  the  "floating"  Milestone  II  in  the  current  DoDD  5000.1. 

2.  Review  and  revise  as  necessary,  the  regulations  that  inhibit  or 
prohibit  the  application  of  procurement  funds  during  Full  Scale 
Engineering  Development,  or  provide  additional  RDT&E  funds  for 
production  preparation. 

3.  Continue  pressing  for  funding  flexibility  -  particularly  for 
authority  to  transfer  funds  between  RDT&E  and  procurement  accounts. 

4.  Improve  Program  Management  by: 

o  Defining  career  progressions  for  prospective  Program  Managers  - 
both  military  and  civilian. 
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o  Enforcing  current  criteria  for  selection  of  Program  Managers. 

o  Assigning  additional  civilians  as  Program  Managers. 

o  Stabilizing  the  tours  of  military  Program  Managers.  Extend 
tours  beyond  the  current  practice  of  about  30  months. 

o  Stopping  the  present  trend  of  assigning  several  programs  to  one 
Program  Manager. 

o  Emphasizing  the  importance  of  colocation  of  functional  directors 
with  the  program  office. 

o  Revising  DoDD  5000.23  to  delete  the  provision  that  encourages 
changing  Program  Managers  at  major  program  milestones. 

o  Expanding  the  Civil  Service  experiment  being  conducted  at  Naval 
Weapons  Center,  China  Lake  and  NOSC,  San  Diego. 

The  panel  also  provided  the  following  recommendations  based  on  the 
observations  previously  discussed: 

1.  Stimulate  the  Standards  and  Specifications  update  program  by  some 
management  attention. 

2.  Clarify  what  Is  meant  by  “Affordability"  and  require  closer  scrutiny 
at  Milestone  II. 

3.  Make  the  Air  Force  Electronic  Systems  Division  report  on  Data 
Requirements  available  to  other  organizations  for  possible 
implementation. 

4.  Consider  expansion  of  the  ASD  Mil-Prime  approach  to  other 
organizations. 
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RESEARCH  ANO 
engineering 


THE  UNDER  SECRETARY  OF  DEFENSE 
WASHINGTON.  D.C.  20301 


28  JUN.1SS2 


MEMORANDUM  FOR  CHAIRMAN,  DEFENSE  SCIENCE  BOARD 

SUBJECT:  Defense  Science  Board  Task  Force  on  the  Transition  of  Weapon 
Systems  from  Development  to  Production 


You  are  requested  to  organize!'  and  convene  a  Defense  Science  Board  Task 
Force  to  review,  evaluate  and  {make  recommendations  concerning  ways  to 
improve  and  accelerate  the  transition  of  weapon  systems  into  production. 

Background  -  The  past  years  have  witnessed  a  dramatic  and  still  acceler¬ 
ating  interest  in  upgrading  the  reliability  and  maintainabiity  of  weapon 
systems,  both  in  and  programmed  for  ultimate  delivery  to  the  Services. 
The  escalating  cost  of  support,  the  unsatisfactory  levels  of  effective¬ 
ness,  and  budget  pressures  to  reduce  the  life  cycle  cost  have  been  major 
stimuli  underlying  this  renewed  concern  for  delivering  reliable  and 
maintainable  weapon  systems.  Yet,  in  spite  of  this  professed  interest 
and  concern,  tangible  improvement  in  the  reliability  and  maintainability 
of  products  reaching  the  field  is  slower  than  desirable.  _  A  major  cause 
seems  to  center  around  the  failure  to  make  the  proper  transition  from 
the  design  and  development  into  manufacturing.  It  is  to  this  element  of 
the  transition  process  that  the  Defense  Science  Board  Task  Force  will 
direct  its  attention. 

Tasks  -  Recent  experience  has  shown  that  when  proper  disciplines  and 
controls  are  applied  throughout  the  conceptual  and  development  process 
in  the  design  and  manufacturing  activities,  a  quality  product  is 
delivered  to  the  field.  The  Task  Force  will  examine  ways  and  methods  to 
more  clearly  define  and  accelerate  this  process  and  will  make 
recommendations  as  appropriate.  These  actions  will  be  directed  toward 
both  the  producing  industry  and  the  administering  government.  The 
following  areas  as  a  minimum  will  be  examined  as  to  their  effects  on  the 
transition  problem  described  previously:. 

1.  Figures  of  merit  for  allocated  design  time  -  examples:  man 

bours/tbousand  components. 

2.  Leading  key  predictors/indicators  of  potential  design  and 

manufacturing  trouble. 

3.  Methods  of  displaying  design  confidence. 

4.  The  design  review  process. 

5.  Test  confidence  vs.  design  confidence. 
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6.  MIL-SPEC  problems  in  speicifying  environmental/ 'operational 
profiles. 

7.  Economics  of  design  confidence  and  of  manufacturing. 

8.  Minimum  requirements  for  manufacturing  capital  investment. 

9.  Shortening  the  test  time  by  integrated  testing. 

10.  Computer-aided-design  requirements  in  the  design  review 
process. 

11.  •  Establishing  part  quality  confidence  in  manufacturing. 

12.  Workmanship  defects  -  cause  and  relation. 

The  findings  and  recommendations  will  be  presented  as  an  interim  report 
by  1  December  1982,  and  in  a  final  report  by  February  1983.  This  Task 
Force  will  be  sponsored  by  the  Deputy  Under  Secreary  of  Defense  for 
Research  and  Engineering  (Acquistion  Management) ,  Mr.  William  A.  long. 
Mr.  Willis  Willoughby  hag  agreed  to  serve  as  chairman.  Mr.  John- Smith 
will  serve  as  the  Executive  Secretary  and  Col.  Wayne  Davis  will  be  the 
DSB  Secretariat  representative  on  the  Task  Force. 
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RECOMMENDATIONS  FOR  CONTINUED  OSD/DSB  ACTION 


The  initial  intent  of  the  study  was  to  focus  solely  on  principles  and  tech¬ 
niques  that  were  consistent  with  current  DoD  policy  and  procedures.  This  was 
done  in  most  cases  and  the  results  of  this  effort  are  reflected  in  the  Trans¬ 
ition  Document  (Appendix  D).  However,  some  of  the  findings  and  conclusions 
from  the  study  point  out  the  need  for  considering  certain  revisions  to  estab¬ 
lished  DoD  policies,  government  regulations  and  congressional  legislation. 
Other  results  from  the  study  indicate  the  need  for  specific  and  separate 
initiatives  within  DoD  to  either  emphasize  established  policies  and  programs 
or  to  sponsor  follow-on  study  tasks  and/or  implementing  action.  A  summary  of 
these  recommended  actions  is  provided  in  this  appendix  together  with  a  page 
reference  to  the  the  applicable  section  of  the  Task  Force  Report  for 
additional  details. 


RECOMMENOED  ACTION 


REPORT  REFERENCE 


Service  T&E  Policy  should  be  amended 
to  encourage  agreements  between 
contractors  and  the  military 
services  to  jointly  participate  in 
DT&E  during  FSED. 

Service  policy  concerning  on-site 
engineering  teams  should  be  added  to 
DoOO  5000.40  and  MIC -STD  785B. 

Qualified  Products  List  (QPL) 
program  should  be  re-established  by 
the  government. 

Uhlform  set  of  screening  require¬ 
ments  should  be  completed  by 
Institute  of  Environmental  Sciences 
and  put  into  use. 


Page  18  -  Military  Participation  in 
FSEb  tests 


Page  19  -  Field  Operational  Perfor¬ 
mance  Feedback  during  the  Early 
Manufacturing  Process 

Page  23  -  Establishing  Parts  Quality 
Confidence  in  Manufacturing 


Page  23  -  Establishing  Parts  Quality 
Confidence  in  Manufacturing 


Services  should  clarify  policy  on: 

-  Government  control  of 
technical  data  package  during 
early  production 

-  Timing  of  multiple  sourcing 
during  early  production 

-  Local  Material  Review  Board 
authority 

-  Constructive  Government 
attitude  during  early 
production  phase 

-  Organic  depot  support  versus 
contractor  support 

-  Product  Improvement  Warranties 

Corrective  action  system  commitment 
and  guidance  should  be  provided  by 
DoD  and  communicated  to  contracting 
organizations. 

Risk  Management  techniques  should  be 
given  wider  publicity  throughout 
government  and  industry. 

Industrial  modernization  incentives 
program  should  continue  to  be 
actively  supported  by  DoD. 

Production  planning  early  in  the 
acquisition  process  should  receive 
strong  emphasis  by  DoD. 


Totally  integrated  CAD/CAM  systems 
should  be  encouraged  and  industry 
should  be  motivated  by  DoD  to  adopt 
and  use  these  systems. 

Data  Base  Technology  for  CAD/ CAM 
systems  should  be  developed  under 
leadership  provided  by  DoD  and  the 
National  Bureau  of  Standards. 

Long  range  strategic  planning  for 
Industrial  facilities  should  be 
developed  by  DoD  and  Industry  should 
be  encouraged  to  do  the  same. 

CAD/CAM  training  should  be  encour¬ 
aged  by  DoD  and  emphasized  and 
implemented  throughout  industry. 


Page  25  -  Cooperative  Participation 
by  government  and  Industry;  and 
Page  26  -  Depot  Organic  Support 


Page  27  -  Corrective  Action  Systems 


Page  29  _  Risk  Management 


Page  32  -  Industrial  Modernization 
Incentives 


Page  33  -  System  Acquisition 
Procedures  and; 

Page  24  -  Recipe  for  Entering 
Production 

Page  34  -  Productivity  and  Design 
Manufacturing  Interaction 


Page  34  -  Data  Base  Technology 


Page  35  -  Corporate  Strategic  Planning 


Page  35  -  Human  Resources 


Action  should  be  taken  on  the 
following  issues: 

-  Recovery  of  interest  expense 
as  a  cost 

-  Recovery  of  CAD/CAM  expense 

-  Independent  research  and 
development 

-  Multiyear  procurement 

-  Competition 

-  Facilities  capital  employed 
(Under absorption) 

-  Foreign  Military  Sales  (OAR 
restrictive  requirements) 

-  Technology  transfer 

-  Manufacturing  research 

-  Productivity  payback 

DoD  policy  on  transition  should  be 
modified  to  view  the  transition  as  a 
phase  rather  than  a  single  milestone. 


DoD  policy  on  Milestone  II  decision 
should  state  that  it  represents  a 
commitment  to  produce  and  deploy  the 
system. 

Acquisition  Strategy  should  be 
clarified  and  receive  additional 
emphasis  by  DoD. 

Program  Management  must  be  improved 
by  actions  such  as  those  recommended 
in  the  Panel  report: 

-  Define  career  progression  for 
program  manager 

-  Enforce  current  critera  for 
program  manager  selection 

-  Assign  additional  civilians  to 
program  management  positions 

-  Stablize  tour  of  military 
program  managers 

-  Stop  trend  to  multi -program, 
program  offices 

-  Emphasize  Importance  of 
colocation  of  functional 
directors  and  program  manager 

-  Change  policy  that  encourages 
shift  In  program  manager 
during  transition  (DoDD 
5000.23) 


Page  36  -  Position  Papers 


Page  40  -  Principle  #1:  "Transition 
Phase"  versus  Production  Milestone 
(and  Recommendations,  Page  51) 

(Note:  See  also.  Page  17,  Initial 
Operational  Test  and  Evaluation). 

Page  42  -  Principle  #2:  Commitment 
to  Produce  and  Deploy  (and 
Recommendations,  Page  51) 


Page  44  -  Principle  #3:  Acquisition 
Strategy  (and  Recommendations,  Page 
51) 

Page  46  -  Principle  #4:  Program 
Managers/Program  Management  (and 
Recommendations,  Page  51) 


-  Expand  civil  service 

experiement  initiated  at  Naval 
Weapons  Center,  China  Lake  and 
NOSC,  San  Diego 

Funding  flexibility  should  continue 
to  be  pursued  as  a  very  tangible  and 
relevant  way  to  improve  the 
transition  process. 


Page  47  -  Principle  #5:  Funding 
Flexibil ity  (and  Recommendations, 
Page  51 ) 


